A colleague of mine asked me to explain the concept of “futility analysis” using the plain languages. The question is triggered by the recent news such as “Solanezumab, Gammagard Trials Survive Futility Analysis” In an Alzheimer trial, it even says from the futility analysis, “there is greater than a 20% statistical probability of success in achieving the primary outcome measure of cognitive function preservation.
During a clinical trial, we can perform interim analysis (or DMC, DSMB review) for three different reasons:
- Interim analysis for safety
1) with pre-specified stopping rule (for example stop the trial if we see # of cases of Serious Adverse Events)
2) without pre-specified stopping rule (rely on DMC members to review the overall safety)
- Interim analysis for efficacy: To see if the new treatment is overwhelmingly better than control - then stop the trial for efficacy
- Interim analysis for futility: To see if the new treatment is unlikely to beat the control – then stop the trial for futility - this is called ‘futility analysis’.
In situations 2 and 3, the criteria for stopping rule for efficacy could be different from the stopping rule for futility, but need to be pre-specified.
An example for futility analysis: in the beginning of the trial, we assumed 65% successful rate for new treatment group and 50% successful rate for control group. We would like to establish the superiority. In the middle of the study, we did an interim analysis. The interim analysis showed 55% successful rate for new treatment group and 50% successful rate for control group. Based on the results from interim analysis, we can calculate the probability and conditional power: if we continue to finish the study, what is the probability of new treatment group better than control? If this probability is too small and meet the pre-specified criteria, we would stop the trial for futility. If this probability is reasonable, we can continue the trial as pre-planned or we can continue the trial with the sample size adjustment (typically increase due to the smaller effect size).
In a paper by Miller et al “Paclitaxel plus Bevacizumab versus Paclitaxel Alone for Metastatic Breast Cancer”, one pre-planned interim analysis and two additional interim analyses were performed and three stopping rules (for safety, for efficacy, and for futility) were pre-specified and evaluated. It is reasonable to assume that none of these stopping rules was triggered since the study was not stopped.
Futility analysis or stopping the trial for futility is not without controversy. An article by Schoenfeld and Meade discussed this issue. See “Pro/con clinical debate: It is acceptable to stop large multicentre randomized controlled trials at interim analysis for futility”.
- Snapinn et al (2006) Assessment of futility in clinical trials
- Lachin (2005) A review of methods for futility stopping based on conditional power (or free access from here)