Tipping point analysis (TPA) is a key sensitivity analysis mandated by regulatory agencies like the FDA to assess the robustness of clinical trial results to untestable assumptions about missing data. Specifically, it explores how much the assumption about the missing not at random (MNAR) mechanism would have to change to overturn the study's primary conclusion (e.g., a statistically significant treatment effect becoming non-significant). See a previous blog post "Tipping point analysis - multiple imputation for stress test under missing not at random (MNAR)"
One-Way Tipping Point Analysis for Robustness Assessment
A one-way tipping point analysis is a sensitivity method used to evaluate the robustness of a study’s primary findings by systematically altering the missing data assumption for only one treatment group at a time—most commonly the active treatment arm. While the missing outcomes in the control group are typically handled under a standard Missing at Random (MAR) or Jump to Reference assumption, the missing outcomes in the active arm are subjected to a varying "shift parameter" (δ). This parameter progressively penalizes the imputed values (e.g., making them increasingly worse) until the statistically significant treatment effect disappears, or "tips." By identifying this specific value, researchers can present a clear, one-dimensional threshold to clinical experts and regulators, who then judge whether such a drastic deviation from the observed data is clinically plausible or an unlikely extreme.
Two-Way Tipping Point Analysis for Robustness Assessment
A two-way TPA is an advanced method to assess robustness by independently varying the missing data assumptions for both treatment groups (e.g., the active treatment arm and the control/reference arm).
Missing Data Assumptions (MAR vs. MNAR)
The two-way TPA is used to assess the robustness of the primary analysis, which is typically conducted under the assumption of Missing at Random (MAR).
Missing at Random (MAR): Assumes that the probability of data being missing depends only on the observed data (e.g., a patient with a worse baseline condition is more likely to drop out, and we have observed the baseline data).
Missing Not at Random (MNAR): Assumes that the probability of data being missing depends on the unobserved missing outcome data itself (e.g., a patient drops out because their unobserved outcome has worsened more than what is predicted by their observed data).
Robustness Assessment
The two-way TPA evaluates robustness to plausible MNAR scenarios. This is done by imputing the missing outcomes (often starting with an MAR method like Multiple Imputation) and then applying a systematic, independent "shift parameter" (or δ) to the imputed values in each arm.
Process: The shift parameters (δActive and δControl) are varied systematically across a two-dimensional grid, typically in a direction that reduces the observed treatment effect.
Tipping Point: The δActive and δControl values at which the primary conclusion (e.g., statistical significance) is "tipped" or overturned define the tipping point.
Robustness: The larger and/or more clinically implausible the combination of shift parameters required to overturn the conclusion, the more robust the original result is considered to be under different MNAR assumptions.
Two-Way Tipping Point Result Tables
The results of a two-way TPA are typically presented as a grid or heat map table where:
One axis represents the shift parameter applied to the missing outcomes in the Active Treatment arm (δActive).
The other axis represents the shift parameter applied to the missing outcomes in the Control/Reference arm (δControl).
The cells of the table contain the resulting p-value or estimated treatment difference for that specific combination of assumptions.
The goal is to find the boundary of the grid where the result crosses the significance threshold (e.g., p >= 0.05 or the lower bound of the confidence interval crosses the null value).
Comparison: One-Way vs. Two-Way Tipping Point Analysis
The choice between one-way and two-way TPA is a trade-off between simplicity and comprehensiveness.
| Feature | One-Way Tipping Point Analysis | Two-Way Tipping Point Analysis |
| Missingness Assumption | The shift parameter (δ) is only applied to one arm, usually the active treatment group, while the missing data in the control arm are imputed based on the MAR assumption (e.g., Jump to Reference). | Independent shift parameters (δActive and δControl) are applied to both arms simultaneously. |
| Sensitivity Explored | Explores MNAR scenarios where dropouts in one arm have systematically worse/better outcomes than assumed by MAR, relative to the other arm's MAR assumption. | Explores a two-dimensional space of MNAR scenarios, allowing dropouts in both arms to vary independently. |
| Complexity | Simpler to calculate and interpret (one dimension). | More computationally intensive and complex to interpret (two-dimensional grid). |
| Plausibility | Often viewed as less comprehensive, as it does not model the possibility of simultaneous, independent MNAR mechanisms in both arms. | Considered more comprehensive as it allows for a wider range of clinically plausible and implausible MNAR scenarios. |
| Result Presentation | A line plot or simple table with a single 'tipping point' value. | A grid/matrix table or heat map showing the boundary of non-significance. |
In essence, the two-way TPA is generally preferred by regulatory agencies for its superior ability to assess robustness because it explores a more realistic and exhaustive range of asymmetric MNAR mechanisms.
