Analysis of Data from Open-Label Extension (OLE) Study

In the previous article, the open-label extension (OLE) study was discussed. The OLE study is usually designed as a separate study from the RCT (the parent study) with its own study protocol and a separate electronic data capture (EDC) system even though all participants in the OLE are rollovers from the parent study. 

When analyzing the data from the OLE study, the data from the parent study often needs to be considered or combined for the analysis. The data can be analyzed with three different baselines:

• Baseline at the beginning of the OLE study - the data from the OLE study will be analyzed separately from the parent study
• Baseline at the beginning of the parent study (at the randomization of the parent study (RCT)) – the data from the OLE study and the parent study are combined and delayed start analysis can be performed to look at the delayed effect or never catch up effect
• Baseline at the first dose of the active drug – to look at the long-term trajectory of safety and efficacy variables for the experimental drug. For participants who were in the active arm of the parent study, the baseline would be at the randomization; for participants who were in the placebo arm of the parent study, the baseline would be at the beginning of the OLE study

If the efficacy outcome is a continuous variable, the delayed start analysis can be performed with the combined data from the parent and the OLE studies. See previous discussions: 

• Demonstrating the disease-modifying effect through delayed start study design or delayed start analyses
• Randomized withdrawal design and delayed start design in rare disease clinical trials
• Randomized Start Design (RSD)

Here are some articles discussing the application of the delayed start analysis in this setting. One of the delayed start analysis approaches is to perform the non-inferiority test to see if the treatment difference observed at the end of the RCT is preserved at the end of the OLE. A non-inferiority margin is pre-defined, the Mixed Models for Repeated Measures (MMRM) method is used to analyze the combined data from the parent study (RCT) and the OLE study, and the results from MMRM analysis are compared to the non-inferiority margin.  

• Lynch et al (2022) Efficacy of Omaveloxolone in Friedreich’s Ataxia: Delayed-Start Analysis of the MOXIe Extension
• Liu-Seifert et al (2014) A Novel Approach to Delayed-Start Analyses for Demonstrating Disease-Modifying Effects in Alzheimer’s Disease
• Liu-Seifert (2015) Delayed-start analysis: Mild Alzheimer’s disease patients in solanezumab trials, 3.5 years 
• Vershuur (2019) Randomized Delayed-Start Trial of Levodopa in Parkinson's Disease - the protocol/SAP
• Cohen et al (2019) An open-label extension study todemonstrate long-term safety and efficacyof ABP 501 in patients with rheumatoidarthritis

Two potential outcomes from the delayed start analyses are meaningful: 

Never catch up:

the placebo group (or delayed start group) will never catch up with the experimental treatment group after switching to the experimental treatment in the OLE study - suggesting the importance of the early treatment with the experimental drug and potential disease-modifying effect. For example, Chapman et al (2015) performed the delayed start analysis using the data from the double-blind trial and the subsequent open-label extension study. The results depicted below indicated the 'never catch up' scenario where the patients in the placebo group were never able to catch up with the AIPI (an enzyme augmentation treatment) group in terms of lung density change from baseline. 

Placebo group catch-up after treatment switching:

The placebo group (or delayed start group) catch up with the experimental treatment group after switching to the experimental treatment in the OLE study - emphasizing the treatment effects observed in the RCT. For example, Rosich et al (2022) performed the delayed start analysis using the data from a double-blind trial and its OLE study for the drug galcanezumab in patients with chronic migraine. After switching to galcanezumab doses at the start of OLE study (at month 3), the previous placebo group experienced a rapid mean reduction of 6.8 migraine headache days within the first month, catching up with the previous double-blind galcanezumab groups by month 4, and then maintaining that reduction over time.

If the efficacy outcome is overall survival (time to death), there is usually an insufficient number of death events from the randomized, controlled parent study for running the log-rank test or Cox regression. However, combining the data from the RCT and the OLE study, there may be enough death events for performing meaningful survival analyses. When performing the survival analysis using the data from the RCT and the OLE study, the start of the OLE study can be considered as treatment crossover or treatment switching - a situation often occurs in oncology clinical trials. Different approaches have been proposed to handle the treatment crossover or treatment switching due to the transition from the RCT to OLE study. These approaches were summarized in EMA's documet "Question and answer on adjustment for cross-over in estimating effects in oncology trials" and in a presentation by Norbert Hollaender (2014) "Methods to estimate survival time after treatment switching in oncology – overview and practical considerations".

Compared to the traditional intention-to-treat approach, it is better to perform the overall survival analyses using more sophisticated methods (such as the rank preserving structural failure time (RPSFT) method and The Inverse Probability of Censoring Weighting (IPCW) to adjust for treatment crossover or treatment switching due to the transition from the RCT to OLE study.  

Open-Label Extension (OLE) Study

I asked OpenGPT to write a blog article about the open-label extension study and got the following response. The contents from OpenGPT seem to be relevant, but not entirely correct. 

"An open-label extension (OLE) study is a type of clinical trial that allows patients who have completed an initial trial to continue receiving the treatment they received during the initial trial. This type of study is beneficial to both the patient and the researcher, as it allows the patient to continue to receive the therapy and the researcher to gain further insight into the effects of the therapy over a longer period of time. 

Open-label extension studies are often used when the initial trial was short-term and the therapy may have long-term effects. This type of study allows researchers to observe the effects of the therapy over a longer period of time and to determine whether the therapy is effective in the long-term. It also allows researchers to observe any possible side effects that may occur over a longer period of time. 

In an open-label extension study, the patient is aware that they are receiving the same therapy that they received during the initial trial. This means that the results of the study are not blinded, as the patient knows what they are receiving. As such, the results of the study may be subject to bias. To reduce the risk of bias, researchers may use a placebo or a control group in the study. 

In addition to the benefits mentioned above, open-label extension studies are also beneficial to patients as they can continue to receive the therapy they received during the initial trial. This is especially beneficial for patients who may not have access to the same therapy outside of the clinical trial. "

Open-label extension (OLE) study is an extension study of a randomized, controlled trial (where the word 'extension' comes from) and is an open-label study (where the word 'open-label' comes from) where all participants receive the experimental drug. OLE study may be sometimes called 'long-term extension study' and is a study following the traditional randomized controlled trial (RCT). All participants in the OLE study are rolled over from the leading RCT - the parent study. 

OLE study can play a useful role in drug development as the sponsors gather additional data on the long-term safety and/or efficacy of the experimental drug, while also giving the RCT participants free access to the drug in development that they are already familiar with after participating in a previous RCT.

In an article by Taylor and Weatherall (2006) "What Are Open-Label Extension Studies For?", the following three purposes were listed for OLE studies:

The first reason is simply to make the (now known to be) effective but as yet unlicensed drug available to participants who were randomized to placebo; this might have been a requirement of the ethics approval or a means of enhancing recruitment to the original RCT. This purpose does not require systematic data collection, and is not a sufficient reason for publishing the results of prolonged observation. 

A second reason is that further, more prolonged observation may disclose adverse effects that were not observed in the original parent RCT. The likelihood of observing such events is low, since the cohorts are almost always too small to reliably detect rare events. In the case of anti-tumor necrosis factor therapies, open-label extension studies failed to detect reactivation of tuberculosis, a problem that was only identified through post-marketing surveillance and national adverse event registries. Even in the case of the early studies of prednisolone in RA, failure to identify significant steroid-induced osteoporosis was more a function of inadequate technology (lack of bone densitometry) than lack of prolonged open-label extension. For example, the study of prednisolone remained randomized for 2 years2. The safety issues do not constitute a sufficient reason for conducting open-label extension studies. 

The third purpose may be to demonstrate continued efficacy of the drug over a longer period of time or to show that participants randomized to receive the active treatment during the open-label phase achieved outcomes similar to those of participants who received the drug from the beginning of the parent RCT.

An OLE study may be designed after a fixed-duration RCT trial where all participants who completed the fixed-duration of treatments in RCT will be eligible for enrolling in the OLE study. The participants in the active arm in the RCT will continue to receive the experimental drug in OLE and the participants in the placebo arm in the RCT will switch to receiving the experimental drug in OLE. This type of design may be called the 'delayed study design' since the participants in the placebo arm will eventually receive the experimental drug, just receive the experimental drug later than those participants in the active arm in the RCT. 

An OLE study may also be designed after an event-driven RCT where participants receive the study drug in various duration and participants will be rolled over to OLE when a non-fatal protocol-defined clinical event occurs or at the end of the study when the total number of events is reached. There will be four different groups of participants who are in the OLE study:

• Participants in Active Drug group had a non-fatal clinical event during RCT and rolled over to OLE
• Participants in Placebo group had a non-fatal clincial event during RCT and rolled over to OLE
• Participants in Active Drug group did not have a clinical event and reached the end of the study in RCT and rolled over to OLE
• Participants in Placebo group did not have a clinical event and reached the end of the study in RCT and rolled over to OLE

OLE studies have been commonly used in rare disease areas where there is an unmet medical need for the disease and in some cases, there are no alternative treatment options after the RCT participants are off the RCT. If I search clinicaltrials.gov using the terms "open-label extension" or "open label extension", there are more than 4000 OLE studies showing up. 

OLE studies may have different durations (usually greater than 1 year). The duration of an OLE study may depend on the success of the RCT (the parent study). If the RCT is a success, the OLE study will be continued for a longer period of time (perhaps until the drug approval and the commercial drug becoming available). If the RCT fails, the OLE study must be discontinued. 

The OLE study is against the concept of the clinical equipoise and statistical equipoise. Equipoise is defined as a state of genuine uncertainty on the relative value (risk-benefit) of two different treatment options being compared in a clinical trial and is the basis for designing randomized controlled trials. The OLE study following an RCT assumes there are benefits in the experimental drug. It is possible to receive criticism when an OLE study is designed while the risk-benefit profile of the experimental drug has not been sufficiently characterized to justify the extension. 

There may be a debate about the OLE study in terms of the utility of OLE study data and the ethics of enrolling participants in the OLE study. From the regulatory standpoint, given the lack of a control arm in the OLE study, the data from the OLE study may provide limited safety and efficacy support to demonstrate the substantial evidences of efficacy and safety. As such, conducting OLE studies is of unclear utility in terms of regulatory decision-making. There may be potential ethical concerns with continuing patients for a prolonged period on an experimental treatment for which there is minimal efficacy data. 

• Ethics of continuing to provide a drug on an open-label extension study for an "unapproved indication"
• An Open-label Extension Study to Evaluate the Safety and Efficacy of Subcutaneous Injections of Pegvaliase (> 40 mg/day Dose) in Adults with Phenylketonuria  
• Micetich (1996) The ethical problems of the open label extension study
• Taylor et al (2014) Ethics of continuing to provide a drug on an open-label extension study for an "unapproved indication"
• 
  
• Consent to open label extension studies: some ethical issue
• Post-Trial Access to Treatment Corporate best practices

We do see that the OLE study data was used to support/strengthen the evidence of the effectiveness of experimental drugs in NDA/BLA submissions. For example, the data from an OLE study, along with the results from a phase 2 RCT, was included in the NDA submission by Amylyx for their drug (Phenylbutyrate–Taurursodiol) in the treatment of Amyotrophic Lateral Sclerosis (ALS). The survival data from both the RCT and the OLE study were analyzed to provide evidence of survival benefits. After two advisory committee meetings, FDA finally approved Amylyx's drug (Relvyrio) for the treatment of ALS based on the efficacy evidence from a small phase 2 RCT supported by the survival data from the combined RCT and OLE study.  

• Amylyx Pharmaceuticals Submits New Drug Application (NDA) for AMX0035 for the Treatment of ALS
• Trial of Sodium Phenylbutyrate–Taurursodiol for Amyotrophic Lateral Sclerosis
• Long-term survival of participants in the CENTAUR trial of sodium phenylbutyrate-taurursodiol in amyotrophic lateral sclerosis

In summary, there are pros and cons to conducting the OLE study following an RCT. 

Pros: 

• Ethical – allow the patients in the experimental drug group to continue the experimental drug or allow the placebo patients to have a chance to receive the experimental drug especially when there is no alternative treatment available
• Collecting the long-term safety data
• Collecting the long-term efficacy data such as the survival data
• If designed appropriately, the data from an OLE study can be used to do “delayed start analysis” (i.e., never catch-up analysis)
• May provide supporting evidence for regulatory decision making

Cons: 

• Against the concept of clinical equipoise
• For diseases that alternative treatments are readily available, there may be no strong ethical reason to provide the experimental study drug treatment through the OLE study
• Limiting the patients to participate in other RCT trials in the same indication
• Resource and cost for conducting OLE studies are not trivial 
• Not considered an adequate and well-controlled (A&WC) study

Rank Preserving Structural Failure Time Model (RPSFTM) to account for treatment crossover

In a previous post "Treatment crossover in parallel-group, randomized, controlled clinical trials", treatment crossover was discussed. Treatment crossover (or treatment switching) occurs when patients switch from their randomized arm to the other treatment during the study. In handling the treatment crossover, the naïve approaches (such as ITT analysis, exclusion of the treatment crossover subjects, and censoring at the time of the crossover) can cause a biased estimate of the treatment difference. More sophisticated approaches are needed to handle the treatment crossover. One of these approaches is called 'Rank Preserving Structure Failure Time Model (RPSFTM)". 

RPSFTM method was proposed by Robins and Tsiatis (1991) in their paper "Correcting for Non-Compliance in Randomized Trials Using Rank Preserving Structure Failure Time Models". The RPSFTM is a method used to adjust for treatment switching in trials with survival outcomes. The method is randomization based and uses only the randomized treatment group, observed event times and treatment history in order to estimate a causal treatment effect. The treatment effect is estimated by balancing counter-factual event times (i.e. the time that would be observed if no treatment were received) between treatment groups. 

We are seeing examples of RPSFTM application in oncology trials (especially open-label randomized trials) and in rare disease clinical trials. 

Hussain et al (2022) published a paper on NEJM "Survival with Olaparib in Metastatic Castration-Resistant Prostate Cancer". The results were from an open-label, phase 3 trial where patients were randomly assigned in 2:1 ratio to receive olaparib or physician's choice of enzalutamide or abiraterone plus prednisone as the control therapy. Patients in the control therapy group were allowed to be crossed over to olaparib after imaging-based disease progression criteria were met. Overall survival was analyzed using the naïve approach (intention-to-treat approach). Sensitivity analysis using the RPSFTM method was then performed to adjust for control patient crossover to olaparib.  Kaplan-Meier plots for the observed data and for crossover-adjusted analysis were depicted below: 

EISAI's Lenvatinib was approved for the treatment of patients with progressive, radioiodinerefractory differentiated thyroid cancer. The NDA approval was based on a pivotal study (Study 303). Study 303 is an international, double-blind, randomized 2:1, placebo-controlled, parallel-group, 2-arm trial. Patients would receive lenvatinib or placebo daily and could be treated until disease progression confirmd by IIR (RECIST v1.1) or unacceptable toxicity. The primary endpoint was progression-free survival with secondary endpoints of ORR and overall survival. Patients randomized to the placebo arm who had confirmed progression could choose to cross over and receive open-label lenvatinib. Overall survival was analyzed with the pooled data from the randomized portion of the study and the optional open-label extension phase. The effect of lenvatinib on overall survival was potentially confounded by the crossover of 83% of patients on the placebo arm to receive lenvatinib in the optional open-label (OOL) extension Phase.

The rank preserving structural failure time (RPSFT) model was then used in OS analysis to correct the bias introduced by cross-over and estimate the true treatment effect on OS. Here is the reference of RPSFT in the FDA's statistical review:

Amylyx's RELYVRIO was approved by FDA for the treatment of ALS. The approval was based on a pivotal study (CENTAUR) and its open-label extension study (CENTAUR-OLE). The pivotal CENTAUR study was a randomized, placebo-controlled, double-blind, 24-weeks study in patients with ALS. Patients who completed 24-week randomized treatment were rolled over to an open-label extension study where all patients received the active drug. Patients in the active drug group in the randomized trial would continue with the active drug in the open-label extension study; patients in the placebo group in the randomized trial would switch or cross over to the active drug. To provide substantial evidence of the effectiveness of the active drug, the sponsor performed the analyses for long-term overall survival with the combined data from the randomized study and the open-label extension study. RPSFTM method was employed to handle the switch or crossover of the placebo patients in the randomized study to active drug in the OLE. Here is the description of the RPSFTM analysis from the briefing book for FDA Adcom. 

The drug Uptravi was approved for the treatment of pulmonary arterial hypertension. The efficacy and safety were based on a pivotal study followed by an open-label extension study. The pivotal study was designed as an event-driven study where patients who had clinical worsening events would be rolled over to the open-label extension study. The placebo patients in the randomized study would switch or cross over the active drug in the open-label extension study. While the randomized study showed the treatment benefit in reducing the risk of clinical worsening events, there was an imbalance in the number of deaths (more death events in active drug group than the placebo group).  The analyses for long-term overall survival with the combined data from the randomized study and the open-label extension study became necessary to mitigate the concern about the imbalance in the number of deaths observed in the randomized trial. RPSFTM method was employed to handle the switch or crossover of the placebo patients in the randomized study to the active drug in the OLE. EMA's assessment report described the RPSFTM analyses. 

An add-on package (RPSFTM) publically available for fitting rank preserving structural failure time models is available for R (Bond and Allison, 2017), and can be installed from the CRAN web portal. A SAS program was written by Bradford J. Danner and Indrani Sarkarto to perform RPSFTM analysis
 

• R-package: https://cran.r-project.org/web/packages/rpsftm/vignettes/rpsftm_vignette.html
• Paper about R-package: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5858764/
• Danner and Sarkarto (2018) Implementing the Rank-Preserving Structural Failure Time Model in SAS® and R 

 Additional References: 

• Hollaender (2014) Methods to estimate survival time after treatment switching in oncology - overview and practical considerations
• Paganoni et al (2022) Survival analyses from the CENTAUR trial in amyotrophic lateral sclerosis: Evaluating the impact of treatment crossover on outcomes
• Latimer et al (2016) Adjusting for treatment switching in the METRIC study shows further improved overall survival with trametinib compared with chemotherapy
• Latimer and Abrams (2014) NICE DSU Technical Support Document 16: Adjusting Survival Time Estimantes in the Presence of Treatment Switching
• Korhonen et al (2012) Correcting Overall Survival for the Impact of Crossover via a Rank-Preserving Structural Failure Time (RPSFT) Model in the RECORD-1 Trial of Everolimus in Metastatic Renal-cell Carcinoma
• White et al (1999) Randomization-based Methods for Correcting for Treatment Changes: Examples from the CONCORDE Trial
• Locke et al (2022) Axicabtagene Ciloleucel as Second-Line Therapy for Large B-Cell Lymphoma
• Bidard et al (2022) Switch to fulvestrant and palbociclib versus no switch inadvanced breast cancer with rising ESR1 mutation duringaromatase inhibitor and palbociclib therapy (PADA-1):a randomised, open-label, multicentre, phase 3 trial
• Turner et al (2018) Overall Survival with Palbociclib and Fulvestrant in Advanced Breast Cancer
• Zhu et al (2021) Final Overall Survival Efficacy Results of Ivosidenib for Patients With Advanced Cholangiocarcinoma With IDH1 Mutation

Treatment crossover in parallel-group, randomized, controlled clinical trials

Randomized, controlled clinical trial (RCT) is the golden standard in drug development. RCTs are usually designed as parallel-group to compare the experimental treatment with a control group (usually the placebo). Eligible patients are randomized to one of the treatment arms (the experimental treatment or placebo). The patients who are randomized to the experimental treatment arm will receive the experimental treatment for the duration of the study and patients who are randomized to the placebo arm will receive the placebo for the duration of the study. 

There are situations where the treatment crossover is allowed by the protocol and the treatment crossover is usually one-sided (i.e., patients on the placebo arm crossed over to the experimental treatment arm, not patients on the experimental treatment arm crossed over to the placebo arm). Treatment crossover can be seen in oncology clinical trials (especially the open-label, randomized trials) or in rare disease clinical trials where the RCT is followed by an open-label extension study. 

In EMA's scientific guidance "Question and answer on adjustment for cross-over in estimating effects in oncology trials", the treatment crossover was described as the following:

In oncology trials, one-sided cross-over of control patients to the experimental treatment may occur, e.g. after progression. No objections from a methodological perspective exist against systematic crossover, where systematic means that there is an objective criterion which determines whether a control patient will cross over to the experimental treatment, or not. One example is when all control patients switch to experimental treatment at the same calendar time (e.g. after an interim analysis declaring superiority); however, this is provided that unconfounded overall survival (OS) data are not considered necessary to evaluate efficacy or safety. Another example of systematic cross-over is when by design a control patient must switch to experimental treatment when that patient experiences progression and the outcome is another measure than progression, e.g. OS, if it is justified to use this design. Nonsystematic cross-over can occur, for instance, when the study protocol allows cross-over after progression at the discretion of the investigator. This document addresses the situation where crossover of control patients is not systematic and there is interest in estimating the effect in the (hypothetical) situation that no cross-over would have occurred in the trial, under the assumption that the experimental treatment cannot introduce harm or deterioration of the condition under investigation in the control patients who cross over. In particular, it should be fully justified that this hypothetical effect is a relevant one for regulatory decision making. It should be noted that due to the uncertainties involved in the methods described below, such estimations should, at present, be used primarily as supportive or sensitivity analyses. 

The guidance defines the treatment crossover as systematic crossover and nonsystematic crossover:

• Systematic crossover is for clinical trials where there is an objective criterion which determines whether a control patient will cross over to the experimental treatment, or not
• Nonsystematic cross-over can occur, for instance, when the study protocol allows cross-over after progression at the discretion of the investigator

Systematic crossover can be seen in the following situations: 

• all control patients switch to experimental treatment at the same calendar time (e.g. after an interim analysis declaring superiority; at the time of study closure)
• individual patients switch to experimental treatment at a different time when patient experiences an event (progression, clinical worsening event, or complete the scheduled treatment duration)
• or a mixture of both situations above

In rare disease areas, the clinical development program usually includes an RCT followed by an open-label study. Because of the rarity of the patients and lack of alternative treatment options, clinical trial participants who complete the randomized portion of the study will be rolled over to an open-label extension (OLE) study where all patients receive experimental treatment. To consider the RCT portion and OLE portion of the study as a whole, the patients in the control arm are crossed over to the experimental treatment arm and the patients in the experimental treatment arm continue the experimental treatment (no crossover). By design, patients who receive the control in the RCT portion of the study cross over to experimental treatment in the OLE portion of the study, which is a perfect example of a systematic crossover. 

Amylyx conducted a phase 2 RCT with a fixed treatment duration (24 weeks) "Trial of Sodium Phenylbutyrate–Taurursodiol for Amyotrophic Lateral Sclerosis". Patients who completed 24 weeks of study treatment (Sodium Phenylbutyrate–Taurursodiol or placebo) were then rolled over a separate open-label extension study where all patients received Sodium Phenylbutyrate–Taurursodiol. Overall survival was analyzed using the combined data from both the RCT and the OLE studies. This can also be viewed as a one-sided crossover where all patients in the placebo arm crossed over to the experimental treatment arm in the OLE. In this case, individual patients switch to experimental treatment at different calendar times, but all after the scheduled RCT duration of 24 weeks. 

The RCT could also be designed as an event-driven study, patients who experienced an event would then be rolled over to the OLE study. Patients who do not experience an event would also be rolled over the OLE study at the RCT study closure when the total number of events was reached. This situation can also be viewed as a one-sided crossover where all patients in the placebo arm crossed over to the experimental treatment arm when they are rolled over to the OLE study. In this case, individual patients who experience an event switch to experimental treatment at different calendar times, but individual patients who do not experience an event switch to experimental treatment at the same calendar time after the total number of events are reached and the study is closed.  See the following examples: 

• Selexipag (ACT-293987) in Pulmonary Arterial Hypertension (GRIPHON)
• Long-term Single-arm Open-label Study, to Assess the Safety and Tolerability of ACT-293987 in Patients With Pulmonary Arterial Hypertension
• Phase III Clinical Worsening Study of UT-15C in Subjects With PAH Receiving Background Oral Monotherapy (FREEDOM-EV)
• An Open-Label, Long-Term Study of Oral Treprostinil in Subjects With Pulmonary Arterial Hypertension

Different approaches can be employed to analyze the data from clinical trials with one-sided treatment crossover. 

EMA's scientific guidance "Question and answer on adjustment for cross-over in estimating effects in oncology trials" mentioned the following methods:

Different statistical methods have been proposed to adjust overall survival for cross-over, including analysis censoring at time of cross-over, Inverse Probability of Censoring Weighting (IPCW), Rank Preserving Structural Failure Time models (RPSFT), and ‘two-stage’ methods.

In principle, these methods can (be adapted to) address different questions by formulating distinct estimands. For example, IPCW estimates the effect of the experimental treatment versus control as if cross-over by control group patients to the experimental treatment was absent but still includes subsequent therapies. Using RPSFT the analyst could choose the estimate to aim at the effect of experimental therapy only (effect of being ‘on experimental treatment’), but in practice the effect of experimental therapy and subsequent therapies (effect of ‘ever being treated’) is often estimated.

In a presentation by Norbert Hollaender "Methods to estimate survival time after treatment switching in oncology– overview and practical considerations", the following simple('naive') methods and complex methods were discussed:

Simple (‘naive’) methods 

• Intent to treat analysis: as randomized and ignoring that some patients switched
• Exclude treatment switchers: small sample size for control group; destroying the randomization; may produce biased results
• Censor switches at time of ‘cross-over’: informative censoring -> results may be biased
• Time-varying treatment variable: No longer a comparison between randomized Treatment vs. Control arm, more difficult interpretation

Complex methods 

• Inverse-probability-of-censoring weighting (IPCW) :
• Switchers are censored at ‘time point of cross-over’, but patients are weighted according to their probability to switch treatment.
• IPCW method artifically increases weights for patients with low probability of treatment switch and decreases weights for patients with high probability of treatment switch
• Rank Preserving Structural Failure Time (RPSFT) Model
• The RPSFTM models the counter-factual or treatment-free event time
• Estimate the survival time gained/lost by receiving active treatment

In practice, for clinical trial data containing patients with one-sided treatment crossover, the overall survival data may be analyzed using both RPSFT and IPCW methods. The results from different methods can then be compared. For example, in EMA's assessment report for Uptravi (selexipag), both RPSFT and IPCW methods were used to evaluate overall survival with the data from the RCT and the OLE studies. 

The applicant presented two analyses to explore the impact of cross-over from the placebo arm and treatment discontinuations in the selexipag arm on the mortality up to study closure. These are a Rank Preserving Structural Accelerated Failure Time Model (RPSFT Model) and an approach using a Marginal Structural Cox Proportional Hazards Model with time-dependent weights according to the Inverse Probability of Censoring Weighting (IPCW) scheme. For both approaches, the RPSFT and Structural Proportional Hazards Model analyses, patients were considered on “active treatment” if they were treated with selexipag or with an agent targeting the same pathway as selexipag. The number of patients in both treatment arms receiving prostacyclin and analogues with the same target as selexipag after study drug discontinuation was similar (40 in the selexipag arm, 44 in the placebo arm). Considering selexipag and agents targeting the same pathway as selexipag as “active treatment”, patients in the selexipag arm were about 85% of their observation time on active treatment and patients in the placebo arm were about 16% on active treatment.

The results RPSFT Model provide a valuable estimate of relative survival on active treatment compared to no treatment of 1.19 with a quite wide 95% confidence interval of (0.56, 2.05).

Using the Structural Proportional Hazards Model with IPCW weighting the estimate for the hazard ratio for death as if all patients had received active treatment compared to the situation if all patients had never received active treatment was 0.92 with a 95% confidence interval of (0.58, 1.47) for the 1 month time intervals, showing a slight advantage for treatment with selexipag. Both estimations with models with longer time intervals show non-significant lower hazard ratios (0.79 and 0.75).

Assessing potential unblinding due to imbalance in side effects through exit questionnaires

The critical features of the RCTs include '(concurrent) control', 'randomization', and 'blinding'. These clinical trial features can minimize the conscious or unconscious biases in the endpoint assessments including both the efficacy and safety assessments.

 
Blinding is a procedure in which one or more parties in a trial are kept unaware of which treatment arms participants have been assigned to, i.e. which treatment was received. On the contrary, unblinding is the process by which the treatment allocation is broken so that one or more parties in a trial become aware of the treatment arms the trial participants are on. To maintain the integrity of the RCT trial, the unblinding should occur only after the completion of the study and after the clinical database has been locked (no further modifications to the clinical data) or only in some special situations (unblinding for data monitoring committee, unblinding of the individual trial participants for SUSAR reporting,...). A couple of previous posts discussed exactly the same issue. 

• Potential Unblinding due to Imbalance in Adverse Event Profiles
• Is blinded study really blinded? - assessment of blinding /unblinding in clinical trials

Blinding is an important aspect of any trial. How a trial was blinded should be accurately recorded in order to allow readers to interpret the results of a study. If blinding is ever broken during a trial on individual participants, it needs to be justified and explained.

To implement and maintain the blinding (or treatment concealment), the clinical trial materials need to be manufactured and packaged in the same way. The control group (typically placebo) will have the same specifications as the testing product in shape, size, color texture, weight, taste, and smell. 

The most difficult part of maintaining the blinding is that the testing product and the control product (usually placebo) may have different side effects. The clinical trial participants and the investigators may be able to guess which treatment group the participants are on. It is quite challenging to defend the integrity of the blinding if two treatment groups do have different side effects or adverse event profiles. 

Amylyx recently ran into this issue and successfully defended the integrity of the blinding for their pivotal study (CENTAUR trial) that was published in NEJM. CENTAUR trial results were the basis for Amylyx's NDA submission. In a rare occurrence, FDA organized two external committee meetings by the Peripheral and Central Nervous System Drugs Advisory Committee (PCNS). 

In the first PCNS adcom meeting on March 30, 2022, the integrity of maintaining the blinding was raised:

"The potential for diarrhea and bitter taste were described in the informed consent, which may have alerted the patients to these symptoms and could have led to functional unblinding during the study. These are potential review issues we have identified which contribute to the uncertainty of the results."

In the second PCNS adcom meeting on September 7, 2022, the sponsor addressed the potential unblinding issue in their briefing book:

The sponsor concluded:

"Based on an exit questionnaire performed at the end of the randomized phase, it asked investigators and participants what treatment arm they were assigned to. Neither study investigators nor participants were able to guess the treatment assignment. The active group was not able to guess their treatment assignment any better than chance, indicating that taste and GI adverse events were not leading to unblinding."

Assessing the blinding through exit questionnaires can be risky though. As described in my previous post "Is blinded study really blinded? - assessment of blinding /unblinding in clinical trials":

"Ideally, in a double-blind trial, it is a good practice to evaluate for both the subjects and investigators whether or not blinding / masking has been preserved. However, in the real world, it is rare in double-blinded clinical trials to include a formal assessment of how well the blinding has been preserved. If the assessment of blinding becomes a routine, I think that many studies will show that subjects/investigators guessed correctly more frequently than they should have done by chance alone. Part of the reason this assessment has not been done often is perhaps the difficulty to explain the study results if the blinding is found to be compromised. It will be extremely difficult to assess the magnitude of the impact on the safety and efficacy evaluation if the blinding/treatment assignment concealment is compromised."

Sample size re-estimation or sample size increase?

Recently, a press release from a biotech company caught my eye. This seems to be an example of the adaptive design with sample size re-estimation, however, it is unusual that the sample size is decreased as usually the sample size re-estimation results in an increase in sample size. 

Bellerophon Therapeutics, Inc. (Nasdaq: BLPH) (“Bellerophon” or the “Company”), a clinical-stage biotherapeutics company focused on developing treatments for cardiopulmonary diseases, announced today that the U.S. Food and Drug Administration (FDA) has accepted the Company’s proposal to reduce the study size for its ongoing registrational REBUILD Phase 3 trial of INOpulse® for the treatment of fibrotic Interstitial Lung Disease (fILD). The new study size of 140 subjects does not impact the trial’s principal objective or endpoints and maintains power of >90% (p-value < 0.01) for the primary endpoint of Moderate to Vigorous Physical Activity (MVPA) based on the effect size observed in Phase 2.

Following the evaluation of baseline MVPA characteristics, as measured by actigraphy, compliance to treatment and review of safety data of the randomized subjects in the ongoing Phase 3 REBUILD study, the trial’s independent Data Monitoring Committee (DMC) supported reducing the target study size from 300 to 140 subjects.

Sample size re-estimation is one type of adaptive design where the sample size can be adjusted during the study based on a prespecified rule. Sample size re-estimation has its special features:  

Group Sequential Design (GSD) and Sample Size Re-estimation

Clinical trials with adaptive design can be in different forms depending on what the adaptations are. Two commonly utilized adaptive designs are group sequential design (GSD) and sample size re-estimation (SSR). Implementation of both GSD and SSR is through the interim analyses conducted by the independent data monitoring committee. In GSD studies, we set a large sample size and hope to stop the trial early due to the overwhelming efficacy, futility, or safety at the interim analyses. In adaptive design with SSR, we start with a small study and possibly increase the sample size post an interim analysis. Both GSD and SSR can achieve the same benefits of reduced sample size and potentially an earlier conclusion. 

Blinded Sample Size Re-estimation and Unblinded Sample Size Re-estimation

In FDA guidance "Adaptive Designs for Clinical Trials of Drugs and Biologics", sample size re-estimation was described in section B "adaptations to the sample size". Blinded sample size re-estimation is based on interim estimates of nuisance parameters such as the standard deviation for continuous outcome measure and overall event rate for discreet outcome measure. The unblinded sample size re-estimation is a type of adaptive design where adaptation is to prospectively plan modifications to the sample size based on comparative interim results. Blinded sample size re-estimation may be conducted by the sponsor statistician while unblinded sample size re-estimation must be through an independent data monitoring committee.  

Sample Size Re-estimation and Sample Size Increase

In clinical trials with prospectively planned sample size re-estimation, the sample size is usually increased. It is very rare that the sample size is decreased after the interim analysis. For adaptive clinical trials with adaptation on sample size (i.e., sample size re-estimation), the initial sample size estimation can be based on more aggressive assumptions that result in a smaller sample size. In the middle of the study, interim analyses are performed and the decision can be made (by independent DMC and through a prespecified rule) whether or not the sample size should be increased. 

In FDA's guidance discussing the Adaptations to Sample Sizes, while the terms 'sample size re-estimation' and 'sample size adaptation' are used, the sample size increase is really implied. 

Sample Size Adaptation and Sample Size Increase by a Fixed Number

The sample size re-estimation or sample size adaptation is really a binary decision. If the decision is to increase the sample size (after the interim analysis), the sample size will be increased by a pre-specified, fixed number, not increased by a number that is based on the observed treatment effect at the interim analysis. 

If the sample size is increased by a very exact level calculated from the observed treatment effect at the interim analysis to bring the conditional power up to a target level, there is a potential to reverse calculate the effect size or to at least make an educated guess about what the effect size is from the interim analysis

This potential for an educated guess about the effect size is a huge issue from the regulatory point of view. This specific concern is discussed in FDA's guidance ""Adaptive Designs for Clinical Trials of Drugs and Biologics".

Finally, there are additional challenges in maintaining trial integrity in the presence of sample size adaptations. For example, sample size modification rules are often based on maintaining the conditional probability of a statistically significant treatment effect at the end of the trial (often called the conditional power) at or near some desired level. In this scenario, knowledge of the adaptation rule and the adaptively chosen sample size allows a relatively straightforward back-calculation of the interim estimate of treatment effect. Therefore, additional steps should be taken to limit personnel with this detailed knowledge so that trial integrity can be maintained.

Randomized withdrawal design in action - Accord trial in Alzheimer's agitation

The biotech company, Axsome Therapeutics, announced the positive results from one of their pivotal phase 3 studies (Accord study). 

Axsome's approved depression drug clears Alzheimer's agitation trial months after Lundbeck-Otsuka duo

The unique side of the Accord study is the use of a randomized withdrawal design. The study was registered on clinicaltrials.gov as "A Double-blind, Placebo-controlled, Randomized Withdrawal Trial to Assess the Efficacy and Safety of AXS-05 for the Treatment of Agitation in Subjects With Dementia of the Alzheimer's Type"

With the randomized withdrawal design, all participants were given the active drug (AXS-05) in a run-in phase in an open-label manner. Then, those patients who responded to treatment during the run-in phase were randomly assigned, in a double-blind manner, to either continue treatment with AXS-05 or switch to a placebo.

According to the sponsor, the basic idea behind this randomized withdrawal study design is to see whether those who initially experience a benefit stop doing so when moved to a placebo, indicating that the therapy itself is effective — as opposed to results being due to a placebo effect. The randomized-withdrawal design of this phase 3 trial simultaneously improved signal detection and mitigated placebo response. 

With the randomized withdrawal design, the sample size was reduced. Total 178 patients with Alzheimer's disease agitation were enrolled into the study run-in phase. 108 patients who achieved a sustained clinical response were then included in the randomized withdrawal period. 

"The ACCORD study was a double-blind, placebo-controlled, multi-center, randomized withdrawal, U.S. trial which treated 178 patients with Alzheimer’s disease agitation. Patients achieving a sustained clinical response after open-label treatment with AXS-05 were randomized (n=108) in a 1:1 ratio to continue treatment with AXS-05 or to discontinue AXS-05 and switch to placebo."

According to an article on evaluate.com:

"When Axsome decided to stop the Accord study of AXS-05 in Alzheimer’s disease agitation early, hopes for that trial took a nosedive. So it was clearly a pleasant surprise today when the company announced that the study had hit. Axsome’s stock opened up 33%, and some investors might be hoping for an earlier-than-expected filing, despite the fact that results from the pivotal Advance-2 study are not due until 2025. Accord had initially been intended as a second pivotal, alongside the previously completed Advance-1, but when the number of agitation events turned out lower than expected, management decided to switch focus to Advance-2. Despite this, Accord met its primary endpoint, time to relapse of agitation, and a key secondary, relapse prevention. One potential fly in the ointment could be Accord’s randomised withdrawal design; it comprised an open-label lead-in phase in which all 178 patients were given AXS-05, and those that had a sustained clinical response to the agent were randomised to either continue treatment or switch to placebo. AXS-05, a combination of dextromethorphan and bupropion, is approved in depression as Auvelity and moving into Alzheimer’s agitation would be an important expansion."

Given that Alzheimer's agitation is a common disease (70% of Alzheimer's disease patients may have agitation), more than one adequate and well-controlled study (or pivotal, confirmatory studies) are needed to demonstrate substantial evidence for effectiveness. Besides the Accord study (with randomized withdrawal design), two additional studies were conducted by the sponsor: ADVANCE-1 trial was a Phase 2/3 study with an active control arm. ADVANCE-2 trial is a phase 3 confirmatory study with the largest sample size (350 patients in a 1:1 randomization ratio). ADVANCE-1 study results had already been announced. ADVANCE-2 study has just started the enrollment. Both ADVANCE-1 and ADVANCE-2 studies were designed as traditional RCT design - randomized, double-blind, placebo-controlled, parallel groups. 

In a clinical program containing multiple pivotal clinical trials, it is appropriate to select different clinical trial designs. In Axsome's Alzheimer's agitation clinical program, a randomized withdrawal design was used in one of the three pivotal trials, and a traditional RCT design was used in the other two pivotal trials. If all these three trials are successful, the evidence for effectiveness will be more substantial and stronger than three studies with the same study design. 

Treatment Emergent AEs (TEAEs), On-Study AEs, On-Treatment AEs, Non-TEAEs

During the clinical trial, the adverse events (AEs) are collected from the signing of the informed consent to the last dose of the study drug plus some follow-time. For statistical analyses of adverse event data, the treatment-emergent AEs (TEAEs) are usually defined. AEs with an onset date at or after the first dose of the study drug will be defined as TEAEs. AEs with an onset date prior to the first dose of the study drug will then be defined as Non-TEAEs. 

For example, the TEAE can be defined as:

"TEAEs are defined as events that start within the day of the first dose of trial treatment until 28 days after the last dose of treatment" in an SAP for an EMD Serono study.

"Treatment-emergent AEs (TEAEs) are defined as AEs that are not present at baseline or represent an exacerbation of a preexisting condition during the treatment period. Therefore, referencing the protocol, TEAEs will be defined programmatically as any AE record with a start date/time on or after the first study treatment administration (greater than or equal to study day 1), inclusive to the end of the study (specifically the EOS visit or ET visit)." in an SAP for a Regeneron's study

The TEAEs can be further defined based on the comparison of the AE onset date with a cut-off date where the cut-off date may be the last dose date or 28 days after the last dose date. 

In FDA's clinical review document for AstraZeneca's asthma drug, the on-study AE and on-treatment AE were defined:

• On-study AE: events with onset between the first-day dosing and the scheduled follow-up visit. 
• On-treatment AE: events with onset between the first day of treatment and the scheduled end of treatment (EOT) or investigational product discontinuation (IPD) visit. 
• Post-treatment AE: events with onset after the on-treatment period defined above

 On-study AEs include all TEAEs - all AEs recorded on or after the first dose date. 

On-treatment AEs are a subset of all TEAEs or on-study AEs. The AEs with an onset date after the cut-off date will be excluded from on-treatment AEs. 

There are some clinical trials with on-treatment AEs defined as the same as traditional TEAEs (i.e., any AEs with an onset date on or after the first dose of the study drug regardless of the cut-off date).

In a recent workshop "Advancing Premarket Safety Analysis" organized by FDA and Duke Margolis Center for Health Policy, the on-study and on-treatment AEs were specifically discussed. Here are the presentation slides for this topic:

The concept of on-treatment AEs has already been implemented in some clinical trials. For example, in a GSK-sponsored trial "A Phase 3a, Repeat Dose, Open-label, Long-term Safety Study of Mepolizumab in Asthmatic Subjects", the primary outcome measure is "Number of Participants With Any On-treatment Adverse Event (AE) or On-treatment Serious AE (SAE)" where On-treatment AEs and on-treatment SAEs are the events occurring on/after the first dose of open-label mepolizumab date and before/on last dose+28 days.

In a BMS trial "A randomized, open-label, phase 3 study of  BMS-936558 vs. Everolimus in Subjects with advanced or metastatic clear-cell renal cell carcinoma who have received prior anti-angiogenic therapy", the on-treatment AEs were defined as the following with a cut-off date of 100 days of the last dose of study treatment.

"On-treatment AEs will be defined as AEs with an onset date-time on or after the DateTime of the first dose of study treatment (or with an onset date on or after the day of first dose of study treatment if time is not collected or is missing). For subjects who are off study treatment, AEs will be counted as on-treatment if event occurred within 100 days of the last dose of study treatment. No “subtracting rule” will be applied when an AE occurs both pre-treatment and post-treatment with the same preferred term and grade."

Defining on-treatment AEs will require specifying a cut-off date and the cut-off date may be different depending on the potential impact of the study drug after the drug discontinuation and the half-life of the investigational products. 

Defining on-treatment AEs may be necessary for studies with the treatment policy estimand where the efficacy data and AE/SAEs are continued to be collected after the study participants have discontinued the study treatment. 

Previous discussions: 

• Adverse events (AE),treatment-emergent adverse events (TEAE), and adverse drug reaction (ADR)
• Adverse Event Collection: When to Start and When to Stop?

Rolling Review, Real time oncology review (RTOR), and Split real time application review (STAR) Program

For New Drug Application (NDA) and Biological License Application (BLA), the usual process is to submit the entire package with different modules at the same time. The submission package will include the quality, CMC, non-clinical study reports, and clinical study reports,... However, there are processes by which the sponsor can submit the submission package piece by piece: rolling review, real-time oncology review, and split real-time application review (STAR) program.

Rolling Review

Rolling review was one of the benefits for drug products with Fast Track Designation. According to FDA's website "Fast Track" 

A drug that receives Fast Track designation is eligible for some or all of the following:

More frequent meetings with the  FDA to discuss the drug's development plan and ensure the collection of appropriate data needed to support drug approval

More frequent written communication from FDA about such things as the design of the proposed clinical trials and use of biomarkers

Eligibility for Accelerated Approval and Priority Review, if relevant criteria are met

Rolling Review, which means that a drug company can submit completed sections of its Biologic License Application (BLA) or New Drug Application (NDA) for review by FDA, rather than waiting until every section of the NDA is completed before the entire application can be reviewed. BLA or NDA review usually does not begin until the drug company has submitted the entire application to the FDA

Fast Track Designation was one of the expedited programs described in FDA's guidance "Expedited Programs for Serious Conditions – Drugs and Biologics". Other expedited programs are breakthrough therapy designation, accelerated approval, and priority review designation. In FDA's guidance, the Fast Track Designation contains the benefit of submission of portions of an application (Rolling Review): 

Here are rolling review examples: earlier this month, Iveric Bio Announces Submission of First Part of NDA for Rolling Review of Avacincaptad Pegol for the Treatment of Geographic Atrophy; in September, 2022, Vertex and CRISPR Therapeutics Announce Global exa-cel Regulatory Submissions for Sickle Cell Disease and Beta Thalassemia

Real-Time Oncology Review (RTOR) Program

For oncology products, FDA's The Oncology Center of Excellence has a program called "real-time oncology review (RTOR)". RTOR facilitates earlier submission of topline efficacy and safety results, prior to the submission of the complete application, to support an earlier start to the FDA’s evaluation of the application. FDA's website "Real-Time Oncology Review" described the details about RTOR program. 

Some companies have utilized this program in hope of expediting their submission/review process. 

In a press release "SpringWorks Therapeutics Announces Data from Phase 3 DeFi Trial Evaluating Nirogacestat in Adult Patients with Progressing Desmoid Tumors at the European Society for Medical Oncology (ESMO) Congress 2022", it stated that their Nirogacestat for R/R desmoid tumors will be filed through real-time oncology review (RTOR) program.

Nirogacestat has received Orphan Drug Designation from the U.S. Food and Drug Administration (FDA) for the treatment of desmoid tumors and from the European Commission for the treatment of soft tissue sarcoma. The FDA also granted Fast Track and Breakthrough Therapy Designations for the treatment of adult patients with progressive, unresectable, recurrent or refractory desmoid tumors or deep fibromatosis. SpringWorks plans to submit a New Drug Application (NDA) to the FDA in the second half of 2022, which will be submitted for review under the FDA’s Real-Time Oncology Review (RTOR) program.

Amgen's Sotorasib was approved by FDA for the first and only targeted treatment for patients with KRAS G12C-mutated locally advanced or metastatic non-small cell lung cancer. The Sotorasib's BLA submission was through RTOR:

"In the U.S., LUMAKRAS was reviewed by the FDA under its Real-Time Oncology Review (RTOR), a pilot program that aims to explore a more efficient review process that ensures safe and effective treatments are made available to patients as early as possible."

 An article in Life Science Leader magazine "FDA's RTOR Program: Draft Guidance & Insights" provides a good summary of the RTOR program.

Split Real-Time Application Review (STAR)

STAR program builds off the Oncology Center of Excellence’s Real-Time Oncology Review (RTOR) program, In the newly passed PDUFA VII for the years 2023 through 2027, a new program called Split real-time application review (STAR) was proposed. According to PDUFA reauthorization performance goal and procedures fiscal years 2023 through 2027, the STAR program was described as the following: 

D. SPLIT REAL TIME APPLICATION REVIEW (STAR) PILOT PROGRAM

FDA will establish a STAR pilot program, which has the goal of shortening the time from the date of complete submission to the action date, in order to allow earlier patient access to therapies that address an unmet medical need. The STAR pilot program will apply to efficacy supplements across all therapeutic areas and review disciplines that meet specific criteria. Accepted STAR applications will be submitted in a “split” fashion, specifically in two parts (with the components submitted approximately 2 months apart).

1. Scope: The STAR program will seek to expedite patient access to novel uses for existing therapies by supporting initiation of review earlier than would otherwise occur and therefore allowing earlier approval for qualified efficacy supplements. This program will apply across all therapeutic areas and review disciplines for applications that meet specific criteria. An application will be considered eligible for STAR if each of the following criteria are met: a. Clinical evidence from adequate and well-controlled investigation(s) indicates that the drug may demonstrate substantial improvement on a clinically relevant endpoint(s) over available therapies. Breakthrough Therapy Designation (BTD) or Regenerative Medicine Advanced Therapy Designation (RMAT) is not required, but above criteria must be met. b. The application is for a drug intended to treat a serious condition with an unmet medical need. c. No aspect of the submission is likely to require a longer review time (e.g., requirement for new REMS, etc.). d. There is no chemistry, manufacturing, or control information that would require a foreign manufacturing site inspection (i.e., domestic site inspections may be allowed if it does not affect the expedited timeframe).

FDA's website "Split Real-Time Application Review (STAR)" described how the STAR program should be operated.

Multiple Endpoints in Clinical Trials (Final) - FDA Guidance for Industry

Today, the FDA finalized its guidance for industry "Multiple Endpoints in Clinical Trials". The draft version of this guidance was issued in 2017. The guidance is intended to help sponsors better understand FDA's current thinking about the issues related to the multiple endpoints and multiplicity issues for multiple endpoints, and different approaches in handling multiplicity issues. The guidance also discussed composite endpoints and multi-component endpoints. 

Typically, an adequate and well-controlled study will include only one primary efficacy endpoint and then multiple secondary efficacy endpoints, and additional exploratory endpoints. Exploratory endpoints are those endpoints for research purposes or for new hypotheses generation and not for the purpose of the product label. Primary and secondary efficacy endpoints can potentially be included on the product label. However, the testing hierarchy and sound approach for multiplicity adjustment must be pre-specified. The Fixed-Sequence Method in the appendix of this guidance seems to be commonly used. With Fixed-Sequence Method, the secondary efficacy endpoints will be tested only if the primary efficacy endpoint is statistically significant. The secondary efficacy endpoints are ranked or ordered based on the importance of the endpoints and the likelihood of getting statistically significant results. The next secondary efficacy endpoint will be tested only if the previous secondary endpoint is statistically significant. The testing hierarchy will stop once the hypothesis test for one of the secondary endpoints is not statistically significant.  

If the sponsor wants to include secondary endpoints in the product label, multiplicity adjustment for secondary endpoints must be included in the statistical analysis plan. 

In the section discussing the co-primary endpoints, "When Demonstration of Treatment Effects on Two or More Distinct Endpoints Is Recommended to Establish Clinical Benefit (Co-Primary Endpoints)", the examples of clinical trials with co-primary endpoints included in the draft version of this guidance were removed from the final guidance. For example, the draft guidance mentioned the following example and the final guidance did not:

Presumably, this is due to the revised FDA guidance "Early Alzheimer's Disease: Developing Drugs for Treatment" and the availability of the integrated scale - Clinical Dementia Rating Sum of Boxes (CDR-SB) Score:

"An integrated scale that adequately and meaningfully assesses both daily function and cognitive effects in early AD patients is acceptable as a single primary efficacy outcome measure. " 

"Common Statistical Methods for Addressing Multiple Endpoint-Related Multiplicity Problems" was included in the body of the guidance in the draft guidance and is now moved to the Appendix: Statistical Methods. The list of methods remains the same and includes the Bonferroni method; the Holm procedure; the Hochberg procedure; prospective alpha allocation scheme; the fixed-sequence method; resampling-based, multiple-testing procedures; gatekeeping testing strategies; and graphical approaches based on sequentially rejective tests. 

As a regulatory agency, FDA is conservative and tries to avoid false conclusions. Without adequate adjustment for multiplicity, the alpha level (type I error rate) can be inflated, and statistically, significant differences may be wrongly declared for an ineffective drug. In the summary of this guidance, FDA concludes: 

Previous Posts:

• Weighted Bonferroni Method (or partition of alpha) in Clinical Trials with Multiple Endpoints
• Co-primary endpoints and multiple primary endpoints