Excessive number of clinical trial protocol amendments due to complex trial design

In a previous blog post "Protocol amendment in clinical trials", I discussed the impact of protocol amendments on the clinical trial performance and cost and the reasons for driving the protocol amendments. Protocol amendments are unavoidable, but we can try to think about the study design and execution proactively to minimize the number of protocol amendments. Sometimes, the excessive number of protocol amendment are driven by Complex Innovative Trial Design or CID in short (for example, adaptive design, basket/umbrella/platform trial design, expansion cohort design, Bayesian design...).

We noticed an extreme case of a clinical trial with the study protocol amended 50 times. This refers to Study P001 (also known as KEYNOTE-001, NCT01295827) by Merck, which was a large, multi-cohort Phase 1 trial with numerous expansion cohorts that supported the initial accelerated approval of pembrolizumab. in Statistical Review and Evaluation, BLA 125514, FDA Center for Drug Evaluation and Research, August 2014, The FDA reviewer noted this high number of amendments while discussing the complexity of the trial design. KEYNOTE-001 was a massive "seamless" adaptive trial that evolved from a traditional Phase 1 dose-escalation study into a large study with multiple expansion cohorts (Part A, A1, A2, B, C, D, etc.) covering different tumor types (Melanoma, NSCLC) and dosing regimens. The "50 times" figure likely includes all global and country-specific amendments up to the time of the BLA submission in February 2014.

The high number of protocol amendments for KEYNOTE-001 was a direct result of its innovative, "seamless" adaptive study design. Initially launched as a standard Phase 1 dose-escalation trial, the study evolved into a massive, multi-cohort trial that eventually enrolled 1,235 patients.

The 50 amendments occurred primarily due to the following reasons:

• Addition of Expansion Cohorts: As early data showed promising results, the protocol was repeatedly amended to add new expansion cohorts for specific tumor types, most notably melanoma and non-small cell lung cancer (NSCLC).
• Sample Size Increases: Striking patient responses led investigators to increase sample sizes within existing cohorts to better evaluate efficacy endpoints like overall response rate (ORR).
• Adaptive Dosing Changes: The protocol was amended to change dosing regimens based on emerging safety and efficacy data. For example, Amendment 7 changed dosing from every two weeks (Q2W) to every three weeks (Q3W), and Amendment 10 shifted all participants to a fixed dose of 200 mg.
• Biomarker Integration: Amendments were used to add co-primary endpoints related to PD-L1 expression after researchers observed its correlation with drug efficacy. This included the validation of a companion diagnostic assay.
• Regulatory Speed: This "seamless" approach allowed Merck to skip traditional Phase 2 and 3 steps for certain indications, leading to the first-ever FDA approval of an anti-PD-1 therapy.

While efficient, the FDA's statistical reviewers noted that such frequent changes (averaging more than one amendment per month during the most active phases) created significant operational and analytical complexity for the trial. The main challenges in analyzing the KEYNOTE-001 trial data, as noted in the FDA's statistical and medical reviews, stemmed from the extreme complexity of a "seamless" design that was modified more than 50 times. 

The primary analytical hurdles included:

• Statistical Integrity and Type I Error Risk: The frequent addition of new cohorts and subgroups—often based on emerging data—increased the number of statistical comparisons. This raised concerns about "multiplicity," where the probability of finding a significant result by chance (Type I error) increases with every new hypothesis tested.
• Operational and Data Management Complexity: Maintaining data quality was difficult when different sites were often operating under different versions of the protocol simultaneously. The FDA noted that this led to potential adherence issues and made it difficult to isolate single cohorts for clean, standalone submissions.
• Shifting Dosing and Regimens: The trial transitioned from weight-based dosing (2 mg/kg or 10 mg/kg) to a fixed dose (200 mg) and changed the frequency of administration (every 2 weeks to every 3 weeks) mid-study. This required complex "pooled analyses" to prove that efficacy and safety were consistent across these varying schedules.
• Biomarker Selection and Validation: The protocol was amended to include a PD-L1 companion diagnostic while the study was already underway. This created a challenge in defining "training" vs. "validation" sets within the same trial population to establish the diagnostic's cutoff levels without introducing bias.
• Lack of a Control Arm: Because the trial was essentially a massive Phase 1 expansion, it lacked a randomized control arm for several indications. This forced reviewers to rely on cross-trial comparisons and historical data, which are inherently more prone to bias than randomized controlled trials (RCTs).
• Patient Selection Bias: The "adaptive" nature allowed for rapid accrual in specific successful cohorts, which, while beneficial for speed, made it difficult to ensure the final patient population was representative of the broader real-world population.

Although the excessive number of protocol amendments, the results from the KEYNOTE-001 resulted in the FDA approval of pembrolizumab in the treatment of multiple tumor types. KEYNOTE-001 study was also the basis for the NEJM article "Seamless Oncology-Drug Development" by Prowell, Theoret, and Pazdur.

Phased clinical trials, seamless clinical trials, phaseless clinical development process

The drug development process encompasses a series of phased clinical trials, typically categorized as phases 0 (optional), 1, 2, 3, and 4. Phases 0, 1, 2, and 3 primarily serve for premarket assessment, while phase 4 focuses on post-marketing evaluation. Phase 1 and 2 trials are often referred to as 'early phase trials,' while phase 3 trials are known as 'late phase trials,' 'pivotal studies,' or 'confirmatory trials.' Notably, the Code of Federal Regulations (Title 21 pertaining to the FDA) does not explicitly describe each phase of clinical trials. Instead, it mandates 'adequate and well-controlled investigations' to substantiate effectiveness. The Code of Federal Regulations does require the specification in IND application form "Identification of the phase or phases of the clinical investigation to be conducted."

The reliance on phased clinical trials can lead to significant delays and cost escalation in the drug development process, potentially impeding timely access to innovative therapies for patients. To address this challenge, incremental innovations, such as adaptive (seamless) clinical trials and cohort expansion designs, have been explored to streamline clinical trial procedures.

The FDA's guidance for industry titled "Demonstrating Substantial Evidence of Effectiveness With One Adequate and Well-Controlled Clinical Investigation and Confirmatory Evidence" acknowledges the evolving landscape. It emphasizes that confirmatory evidence regarding effectiveness may not solely derive from clinical trials but can also encompass other sources like natural history evidence, real-world data, and evidence from expanded access programs. In certain scenarios, the conventional phased approach to clinical development may not be universally applicable. Embracing a phaseless clinical development process may offer a more pragmatic and suitable alternative in specific contexts.

Phased Clinical Trials

In a previous discussion, we delved into the realm of phased clinical trials, exploring the sequential stages denoted as Phases 0, 1, 2, 3, and 4.

Seamless (phases) Clinical Trials

The concept of seamless phases in clinical trials involves the integration of two distinct phases, such as seamless phase 1/2 trials or seamless phase 2/3 trials. It's worth noting that the FDA's guidance document titled "Adaptive Designs for Clinical Trials of Drugs and Biologics" has evolved beyond the specific term "seamless design." Instead, it incorporates seamless elements within the broader framework of "Adaptations to Treatment Arm Selection." This approach encompasses not only dose selection but also the confirmation of efficacy for the selected dose within a single study.

When registering clinical trials on ClinicalTrials.gov, it is necessary to specify the phases of the trial. Categories such as "Phase 1/Phase 2" and "Phase 2/Phase 3" are utilized to denote seamless designs, reflecting the integration of multiple phases within a single trial protocol. There is no option for "Phase 1/Phase 3" and 'Phase 1/2/3'.

The utilization of expansion cohorts design has gained significant traction within oncology drug development and is now widely recognized as a specialized variant of seamless design. Pioneered by Merck and described in NEJM paper by Prowell et al. in 2016, this approach has garnered attention for its potential to streamline the development process for oncology drugs.

In 2022, the FDA released guidance specifically addressing the use of expansion cohorts, titled "Expansion Cohorts: Use in First-In-Human Clinical Trials to Expedite Development of Oncology Drugs and Biologics Guidance for Industry." This guidance outlines best practices for incorporating expansion cohorts into early-phase clinical trials, with the aim of accelerating the development of oncology therapies.

Studies employing expansion cohorts design may be categorized as "Phase 1/Phase 2" or "Phase 1/Phase 3" trials, depending on the primary objectives of the expanded cohorts. If the focus is on assessing anti-tumor activities, the study may be labeled as a "Phase 1/Phase 2" trial. Conversely, if the expanded cohorts are intended to evaluate efficacy endpoints, the study may be classified as a "Phase 1/Phase 3" trial.

In their 2018 paper "Advancing Clinical Trials to Streamline Drug Development," Bates et al. introduced a notable departure from the traditional clinical trial progression. Instead of adhering to the sequential phases of safety evaluation in Phase 1, efficacy assessment in Phase 2, and comparative efficacy testing in Phase 3, they advocated for the adoption of expansion cohorts. This innovative approach created what they termed a "continuum" or "phaseless" trial model, characterized by the seamless integration of various trial components through the use of protocol amendments and ongoing discussions with the FDA.

By embracing expansion cohorts and the phaseless trial concept, the drug development process could expedite the delivery of new therapies to patients. Moreover, this approach showcased the FDA's willingness to embrace flexibility and innovation in regulatory practices.

A particularly intriguing aspect of this "continuum" expansion cohort model is its circumvention of the conventional drug development paradigm and the intricate regulatory framework that often poses challenges for both investigators and regulators. This departure from the norm represents a significant shift towards a more agile and patient-centric approach to drug development, potentially opening avenues for greater efficiency and accessibility in the advancement of medical therapies.

Phaseless Drug Development Process

In certain contexts, traditional phased clinical trials may not be applicable, particularly in the development of therapies for ultra-rare diseases or in the realm of gene therapy. This divergence from the traditional paradigm has led to the emergence of the term "phaseless" to signify scenarios where the sequential numbering of clinical trial phases becomes less relevant.

There are two primary scenarios where the phaseless approach is observed:

• Single-Trial Clinical Development: In this scenario, the entire clinical development program revolves around a single clinical trial, regardless of whether it would conventionally be labeled as phase 1, 2, or 3. Regulatory approval, such as a New Drug Application (NDA) or Biologics License Application (BLA), may rely on the data from this singular study, which can be supplemented by various other types of evidence outlined in FDA guidance documents.

  An example of this approach is seen in the development of the first CRISPR-Cas9 gene-edited therapy by Vertex/CRISPR Therapeutics for severe sickle cell disease (SCD). FDA approval for this product was based on a single-arm, open-label, multi-site, single-dose Phase 1/2/3 study.

• Seamless Phase 1/2/3 Trials: Alternatively, a phaseless approach can involve the integration of all three traditional phases into a single seamless clinical trial. Pfizer/BioNTech's COVID-19 vaccine study serves as an illustrative example of this approach. Their study, labeled as phase 1/2/3, encompassed a randomized, placebo-controlled, observer-blind, dose-finding investigation to identify preferred vaccine candidates and dose levels in phase 1, followed by an expanded cohort and efficacy assessment in phase 2/3.

Phases of clinical trials become vague nowadays and there is a growing sentiment that the reliance on the numbered phases of clinical trials may become outdated and potentially misleading in the era of innovative trial designs. Instead, regulatory drug approval processes should prioritize the totality of evidence, which can stem from a combination of exploratory and confirmatory trial designs, as well as other sources such as natural history data, real-world evidence, expanded access studies, and animal models. In this context, the numbering of clinical trial phases becomes less crucial, emphasizing the need for a more holistic and adaptable approach to evaluating the efficacy and safety of medical interventions.