Survivorship Bias and its Occurrences in Clinical Trials

Survivorship bias or survival bias is the logical error of concentrating on entities that passed a selection process while overlooking those that did not. This can lead to incorrect conclusions because of incomplete data. Survivorship bias is a selection bias that occurs when an individual only considers the surviving observation without considering those data points that didn’t “survive” in the event. It can lead to incorrect or misleading conclusions.

 
Most famous example of survivorship bias came from the World War II time. During World War II, the high rate of planes being shot down posed a significant challenge. To address this, a team was tasked with improving the protection and armor of the aircraft to reduce their vulnerability. The team analyzed the planes that returned from missions, documenting the locations of damage and creating a damage map. They observed heavy damage on the wings and tail sections, leading them to reinforce these areas. However, despite these modifications, the rate of plane losses did not significantly decrease.

The breakthrough came when the team realized a critical oversight: they were only examining the planes that had successfully returned, meaning the damage they observed was survivable and not the kind that would cause a plane to be lost. This insight shifted their perspective, highlighting that the most vulnerable areas were likely those that hadn’t been hit on the returning planes—areas where damage would be fatal. By focusing on reinforcing these previously overlooked sections, the team was able to better protect the aircraft and pilots, ultimately saving lives. This story underscores the importance of considering not only the visible data but also the hidden or missing information, as what you don’t know can be just as critical as what you do know.

Survivorship Bias in Clinical Trials. 

Survivorship bias occurs when we focus only on the "survivors", "completers", "responders", or successful outcomes in a dataset while ignoring those that didn’t make it. This can lead to overly optimistic or inaccurate conclusions because the full picture isn’t being considered. In the context of clinical trials, survivorship bias can distort our understanding of a treatment’s effectiveness or safety by excluding data from participants who dropped out, didn’t respond to the treatment, or experienced adverse events.

Survivorship Bias are very common in clinical trials even though the term "survivorship bias" may not be explicitly used and may be described under "selection bias".

How Does Survivorship Bias Manifest in Clinical Trials?

1. Dropout Rates and Missing Data
   Clinical trials often experience participant dropouts due to side effects, lack of efficacy, or personal reasons. If researchers only analyze data from participants who completed the trial, they may overestimate the treatment’s effectiveness or underestimate its risks. For example, a drug might appear highly effective because only the patients who benefited from it stayed in the trial, while those who didn’t respond or experienced severe side effects left.
2. Selective Reporting
   Researchers or sponsors may inadvertently (or intentionally) focus on positive outcomes while downplaying or omitting negative results. This can create a skewed perception of a treatment’s success. For instance, if a trial reports only the patients who improved and ignores those who didn’t, the treatment may seem more promising than it actually is.
3. Long-Term Follow-Up Gaps
   Many clinical trials focus on short-term outcomes, which can miss long-term effects. Patients who experience adverse events or relapse after the trial ends may not be included in the final analysis, leading to an incomplete understanding of the treatment’s safety and efficacy.
4. Population Selection
   Clinical trials often exclude certain populations, such as older adults, pregnant women, or individuals with comorbidities. While this is sometimes necessary for safety or feasibility, it can create a biased sample that doesn’t reflect the real-world population. The "survivors" in this case are the participants who met the strict inclusion criteria, potentially limiting the generalizability of the results.

Real-World Consequences of Survivorship Bias

Survivorship bias in clinical trials can have serious implications for patients, healthcare providers, and policymakers. For example:

• Overestimation of Treatment Efficacy: If a drug appears more effective than it truly is, patients may be prescribed a treatment that doesn’t work for them, wasting time and resources.
• Underestimation of Risks: Ignoring data from participants who dropped out due to side effects can lead to an incomplete understanding of a treatment’s safety profile.
• Misguided Policy Decisions: Policymakers relying on biased trial results may approve treatments that aren’t as effective or safe as they seem, potentially putting public health at risk.
  

How Can We Address Survivorship Bias?

1. Intent-to-Treat Analysis or Treatment Policy Strategy in Estimand Framework
   One of the most effective ways to mitigate survivorship bias is to use an intent-to-treat (ITT) analysis or treatment policy strategy in handling the intercurrent event, which includes all participants who were randomized in the trial, regardless of whether they completed it. This approach provides a more realistic picture of the treatment’s effectiveness in real-world conditions.
2. Avoid performing the analyses only on completers
3. Encourage the patients with intercurrent events to remain in the study to minimize the dropouts
4. Transparency in Reporting
   Researchers should report all outcomes, including dropouts, adverse events, and negative results. Journals and regulatory agencies can encourage this by requiring comprehensive data disclosure.
5. Long-Term Follow-Up
   Extending the follow-up period can help capture long-term outcomes and provide a more complete understanding of a treatment’s benefits and risks.
6. Diverse Participant Populations
   Including a broader range of participants, such as older adults and individuals with comorbidities, can improve the generalizability of trial results and reduce bias.
7. Independent Oversight
   Independent review boards and data monitoring committees can help ensure that trials are conducted and analyzed objectively, minimizing the risk of bias.

Conclusion

Survivorship bias is a subtle but significant issue in clinical trials that can distort our understanding of medical treatments. By focusing only on the "survivors", "completers", "responders", or successful outcomes, we risk overlooking critical data that could impact patient care and public health. Addressing this bias requires a commitment to transparency, rigorous analysis, and inclusive research practices. As we continue to advance medical science, it’s essential to remember that what we see isn’t always the full picture—and that the missing pieces may hold the key to better, safer, and more effective treatments.

By being aware of survivorship bias and taking steps to mitigate it, we can ensure that clinical trials provide the most accurate and reliable evidence possible, ultimately improving outcomes for patients everywhere.

Further Reading:

• Elston (2021) Survivorship bias. Journal of the American Academy of Dermatology
• Francesco Pasqualetti (2023) The impact of survivorship bias in glioblastoma research. Critical Reviews in Oncology/Hematology
• Mark É. Czeisler (2021) Uncovering survivorship bias in longitudinal mental health surveys during the COVID-19 pandemic. Epidemiology and Psychiatric Sciences
• Conner (2019) Survivorship Bias in Analyses of Immune Checkpoint Inhibitor Trials. JAMA oncology
• Selection Bias: Selection Bias: The Art of Choosing the Right Data
• WHO (2019) WHO consolidated guidelines on drug-resistant tuberculosis treatment