Futility Analysis in Clinical Trials - Stop the trial for futility

A colleague of mine asked me to explain the concept of “futility analysis” using plain languages. The question is triggered by the recent news such as “Solanezumab, Gammagard Trials Survive Futility Analysis” In an Alzheimer trial, it even says from the futility analysis, “there is greater than a 20% statistical probability of success in achieving the primary outcome measure of cognitive function preservation.

During a clinical trial, we can perform interim analysis (or DMC, DSMB review) for three different reasons:

1. The interim analysis for safety

1)       with pre-specified stopping rule (for example stop the trial if we see # of cases of Serious Adverse Events)

2)       without pre-specified stopping rule (rely on DMC members to review the overall safety)

2. The interim analysis for efficacy: To see if the new treatment is overwhelmingly better than control  - then stop the trial for efficacy
3. The interim analysis for futility:  To see if the new treatment is unlikely to beat the control – then stop the trial for futility  - this is called ‘futility analysis’.

In situations 2 and 3, the criteria for stopping rule for efficacy could be different from the stopping rule for futility, but need to be pre-specified.

In situation #2 (stopping the study for efficacy), there will be a penalty for alpha spending - if the overall alpha is 0.05, a portion of the alpha will be allocated for tee interim analysis and the alpha for final analysis will be less than 0.05.   In situation #3, there is no penalty for alpha spending. 

An example for futility analysis: at the beginning of the trial, we assumed 65% successful rate for new treatment group and 50% successful rate for the control group. We would like to establish superiority. In the middle of the study, we did an interim analysis. The interim analysis showed 55% successful rate for new treatment group and 50% successful rate for the control group. Based on the results from the interim analysis, we can calculate the probability and conditional power: if we continue to finish the study, what is the probability of the new treatment group better than control? If this probability is too small and meets the pre-specified criteria, we would stop the trial for futility. If this probability is reasonable, we can continue the trial as pre-planned or we can continue the trial with the sample size adjustment (typically increase due to the smaller effect size).

In a paper by Miller et al “Paclitaxel plus Bevacizumab versus Paclitaxel Alone for Metastatic Breast Cancer”, one pre-planned interim analysis and two additional interim analyses were performed and three stopping rules (for safety, for efficacy, and for futility) were pre-specified and evaluated. It is reasonable to assume that none of these stopping rules was triggered since the study was not stopped.  

Futility analysis or stopping the trial for futility is not without controversy. An article by Schoenfeld and Meade discussed this issue. See “Pro/con clinical debate: It is acceptable to stop large multicentre randomized controlled trials at interim analysis for futility”.

Further readings:

• Snapinn et al (2006) Assessment of futility in clinical trials
• Lachin (2005) A review of methods for futility stopping based on conditional power (or free access from here)