Basket Trial: challenges and disadvantages

With the advances in biomarker identification and precision medicine, the biomarker-based clinical trial design becomes a new trend. In 2017, FDA approved Merck's Keytruda (pembrolizumab) as the first cancer treatment for any solid tumor with a specific genetic feature (for the treatment of adult and pediatric patients with unresectable or metastatic solid tumors that have been identified as having a biomarker referred to as microsatellite instability-high (MSI-H) or mismatch repair deficient (dMMR). 

A special class of biomarker designs is "master protocols” which includes Basket trials, umbrella trials, platform trials.

• Master Protocol: An over-arching protocol or trial mechanism comprised of several parallel sub-trials differing by molecular features or other objectives.
• Basket Trial: A master protocol where each sub-trial enrolls multiple tumor types ("the basket"). According to NCI definition, basket trial is defined as "A type of clinical trial that tests how well a new drug or other substance works in patients who have different types of cancer that all have the same mutation or biomarker. In basket trials, patients all receive the same treatment that targets the specific mutation or biomarker found in their cancer. Basket trials may allow new drugs to be tested and approved more quickly than traditional clinical trials. Basket trials may also be useful for studying rare cancers and cancers with rare genetic changes. Also called bucket trial."
• Umbrella Trial: A master protocol where all patients (and all sub-trials) share a common tumor type ("the umbrella"). According to NCI definition, the umbrella trial is defined as "A type of clinical trial that tests how well new drugs or other substances work in patients who have the same type of cancer but different gene mutations (changes) or biomarkers. In umbrella trials, patients receive treatment based on the specific mutation or biomarker found in their cancer. The drugs being tested may change during the trial, as new targets and drugs are found. Umbrella trials may allow new drugs to be tested and approved more quickly than traditional clinical trials."
• Platform Trial: A master protocol where sub-trials may be added or removed in an operationally seamless way. I-SPY trials are good examples of a platform trial. 

In industry especially the biotechnology companies, it is not easy to implement the umbrella trial or platform trial without collaborating with other sponsors or partners. The basket trial design is the one that may be more practically implemented. 

While the application of master protocols and basket trials is mainly limited in the oncology trials, there have been some discussions in other areas such as clinical trials in arthritis and rare diseases. 

The popularity of the master protocol and basket trial reminds me of the adaptive design about 15 years ago. The advantages of the new trial designs were over-emphasized and the limitations/disadvantages were de-emphasized. At one time, I had to explain to our senior management why each of our clinical trials was not a good candidate for adopting the adaptive design. Now, if we work on the oncology area, we may face a similar situation and maybe asked why the master protocol and basket trial design are not used. 

For a clinical development program for drugs/biologicals in the oncology area, we will need to evaluate the genetic biomarkers and consider the basket trial design after fully evaluating the pros and cons of using such a design. 

Recently, there have been a lot of discussions about the advantages and disadvantages of the basket trial design.

In a paper by Renfro and Sargent "Statistical controversies in clinical research: basket trials, umbrella trials, and other master protocols: a review and examples", the advantages and disadvantages of the basket trial were discussed.

Advantages of basket trials
Basket trials have several advantages. First, they can provide access to molecularly targeted agents for patients across a broad range of tumor types, potentially including those not otherwise studied in clinical trials of targeted therapies. Secondly, in many cases, molecular testing is carried out locally and confirmation by a central assay is not required before patient enrollment, though tumor and plasma are often banked for subsequent companion diagnostic testing and validation. This feature reduces the time between initial diagnosis and/or eligibility confirmation and later cohort assignment and initiation of treatment. Thirdly, cohorts within basket trials are often small and utilize single-stage or two-stage designs, which yield quick results, given sufficient accrual.

Limitations of basket trials
One major limitation of basket trials is the assumption that molecular profiling may be sufficient to replace histological tumor typing, as, in some cases, histological tumor type has been found to predict response to treatment more strongly than the biomarkers or mutations comprising the studied cohorts. Even outside the context of a basket trial, it was recognized that V600E BRAF-mutant melanoma or hairy cell leukemia are responsive to BRAF inhibition, while colon tumors with the same BRAF mutation are not. This issue may be anticipated, as it is well accepted that the environment and location in which a tumor develops may impact its mutational profile as well as differentially predict treatment response across similar profiles. To this end, many have noted that current clinical evidence is insufficient to conclude that molecular descriptors should replace histological tumor typing, and it has been suggested that future studies integrate anatomic with mutational and functional molecular profiling through the use of proteomic technologies and explore multi-gene signatures with combination therapies.

In an ASA webinar "Basket Trials: Features, Examples, and Challenges" by Lindsay Renfro, the advantages and disadvantages were listed as the following: 

Basket Trials: Advantages: 

• Operational efficiencies compared to designing and conducting individual targeted trials without shared infrastructure
• Relatively small sample size per sub-study
• Increased "hit rate" by enrolling patients with rare molecular features across tumor types
• Array of novel therapeutics offered to a broader group of patients who may benefit

Basket Trials: Disadvantages

• Prognostic heterogeneity across tumor types
• Single arm sub-studies generally require a tumor response rate endpoint (with a high bar)
• Challenging to define historical controls across diseases. For this reason, time-to-event endpoints (though often relevant) usually not primary
• Practical challenges with screening may arise

In a recent webinar "Trial Design Concept for a Confirmatory Basket Trial - Dr. Robert Beckman", Dr Bob Beckman discussed the features of the basket trial and then listed 13 challenges (or disadvantages) of the basket trial: 

Features of the Basket Design:

• Tumor histologies are grouped together, each with their own control group (shared control group if common SOC)
• Randomized control is preferred. Single arm cohorts with registry controls may be permitted in exceptional circumstances as illustrated by Imantinib B225 and others
• In an example of particular interest, each indication cohort (each sub-study) is sized for accelerated approval based on a surrogate endpoint such as progressive free survival (PFS) - this may typically be 25-30% of the size of a phase 3 study
• In another approach, an interim evaluation of partial information on the definitive endpoint may be used
• Initial indications are carefully selected as one bad indication can spoil the entire pooled result
• Indications are further "pruned" if unlikely to succeed, based on 1) external data (maturing definitive endpoint from phase 2; other data from class); 2) internal data on surrogate endpoint OR partial information on definitive endpoint
• Sample size of remaining indications may be adjusted based on pruning
• Type I error threshold will be adjusted to control type I error (false positive rate) in the face of pruning. Pruning based on external data does not incur a statistical penalty.
• Study is positive if pooled analysis of remaining indications is positive for the primary definitive endpoint. 1) remaining indications are eligible for full approval in the event of a positive study; 2) full pooling chosen for simplicity; 3) Some of the remaining indications may not be approved if they do not show a trend for positive risk benefit as judged by definitive endpoint. 

Challenges of the Basket Design

Challenge #1: Having a Control Group

In some settings, a control group is not ethical

Resolution: randomized trial should be applied, if and only if:

• There is a clinical equipoise between the two randomized arms
• Experimental agent is not expected to be transformational, only beneficial
• There is a standard of care (SOC) for control:   Example: steroids +/- rituximab for refractory autoimmune diseases
• Current generation of non-randomized basket studies for transformational agents provides SOC baseline for future randomized trials

Challenge #2: Risks of Pooling

One of more indications can lead to a failed study for all indications in a basket

Histology can affect the validity of a molecular predictive hypothesis, in ways which cannot always be predicted in advance

Vemurafenib is effective for BRAF 600E mutant melanoma, but not for analogous colorectal cancer (CRC) tumors

This was not predicted in advance but subsequently feedback loops leading to resistance were characterized

Basket trials are recommended primarily after there has been a lead indication approved (by optimized conventional methods) which has validated the drug, the predictive biomarker hypothesis, and the companion diagnostic.               -    Example, melanoma was lead indication preceding Brookings trial proposal in V600E mutant tumors

Indications should be carefully selected

Indications should be pruned in several steps before pooling

Challenge 3: Different Indications May Have Different Endpoints

Less of an issue for oncology

Even in auto-immune diseases, generalized interim endpoints can be created across diverse diseases:

Interim: improvement (response)

Final: time to worsening

Challenge 4: Timescales of endpoint development may differ

Resolution:

What matters is relative improvement

If necessary, TTE data may be normalized to the medians on control arms of the different indications

Study completes when data is mature on all arms

Challenge 5: SOC may differ between arms

Resolution:

What matters is relative improvement in a redefined disease entity based on a molecular biomarker

Safety must be assessed both as an individual analysis relative to individual control and as a pooled analysis relative to pooled control

Safety data should be available from reference indications and from phase 2 studies

Challenge 6: Threshold for Approval May Different Between Arms

Resolution: study is judged by pooled result of relative improvement with statistical and clinical significance

Thresholds for such criteria are well known

Challenge 7: Clinical validity of the predictive biomarker hypothesis

The clinical validity of the predictive biomarker can only be verified by inclusion of “biomarker negative” patients in the confirmatory study

Addressing the challenge

Recommend a smaller pooled, stratified cohort for biomarker negative patients, powered on surrogate endpoint

Would need to expand the biomarker negative cohort (to evaluate definitive endpoint) if surrogate endpoint shows possible benefit

Prior evidence should permit this if

An approved lead indication has already provided clinical evidence for the predictive biomarker hypothesis

Prior phase 2 studies support the predictive biomarker hypothesis in other indications

Challenge 8: Adjusting for Pruning

Pruning indications that are doing poorly on surrogate endpoints may be seen as cherry picking

This can inflate the false positive rate, an effect termed “random high bias”

Addressing the challenge:

Emphasize use of external data, especially maturing Phase 2 studies, for pruning

Pruning with external data does not incur a penalty for random high bias

Applying statistical penalty for control of type I error when applying pruning using internal data

Methods for calculating the penalty are described in stat methods papers

Rules for applying penalty must be prospective

Penalty is not large enough to offset advantages of design

Challenge 9: Strong Control of FWER

This problem is still open

Challenge:

One or more strongly positive indications can drive an overall pooled positive result and negative indications are carried along

Simulation involves a large number of cases and the degree to which active indications are active affects the results.

A recent MSKCC study simulated a popular Bayesian basket trial design (using a Bayesian hierarchical model) and found FWER of up to 57%.

Authors advocate characterization of FWER by simulation

Challenge 10: Availability of tissue

Tissue sampling and processing are variables that can greatly affect the outcome of a study based on a predicative biomarker

Basket studies will require cooperation and uniformity across departments organized by histology

Addressing the challenge:

The sponsor must have extensive contact with the pathology department and relevant clinical departments at all investigative sites and provide standard methods for tissue sampling, handling, and processing

The sponsor should engage an expert pathologist who is dedicated to training prior to trial start, and troubleshooting during the trial

Challenge 11: High Screen Failure Rate

Pro: patients will have access to tailored therapy

Con: patient has a high risk of being a screen failure if biomarker positive subgroup is low prevalence

Addressing the challenge:

Study should provide a broad-based test like HGS which will give the patient some guidance on alternative therapies if they are screen failures for basket study

Challenge 12: Interim endpoints may not predict definitive endpoints

Addressing the challenge:

Prefilter indications based on maturing definitive endpoint data from phase 2 or other external data

Require consistent trend in definitive endpoints for final full approval

Challenge 14: The Standoff

Health authorities “understandably” won’t commit until given a real example to consider

Sponsors “understandably” cautious about being first to innovate in confirmatory space

Resolution:

FDA, under PDUFA VI pilot program, will be engaging with selected sponsors to bring forward complex innovative designs

We must take this risk for our patients.

Further Readings:

• Renfro and Sargent, Ann Oncol (2017) "Statistical controversies in clinical research: basket trials, umbrella trials, and other master protocols: a review and examples"
• Woodcock and LaVange, NEJM (2017) "Master Protocols to Study Multiple Therapies, Multiple Diseases, or Both"
• Janiauda, Serghioua, and Ioannidisa (2019) "New clinical trial designs in the era of precision medicine: An overview of definitions, strengths, weaknesses, and current use in oncology"
• FDA Modernizes Clinical Trials with Master Protocols - podcast

• FDA draft guidance for industry (2019) Master Protocols: EfficientClinical Trial DesignStrategies to ExpediteDevelopment of OncologyDrugs and Biologics
• NCI-MATCH Trial (Molecular Analysis for Therapy Choice)