Minimizing the missing data: data collection after subjects withdraw from the clinical trial

Recently, I have heard quite some discussions about the data collection in clinical trials for subjects who discontinue early from a clinical trial. The data collection for key efficacy and safety outcomes will continue even after the subjects have discontinued the study treatment, or discontinued from the study. This has been considered as one of the approaches for minimizing the missing data.

In 2000, National Academies published the report “The Prevention and Treatment of Missing Data in Clinical Trials”. The report considered ‘continuing data collection for dropouts’ as one of the key approaches in minimizing the missing data. It had a lengthy narratives about this approach in the report.

CONTINUING DATA COLLECTION FOR DROPOUTS

Even with careful attention to limiting missing data in the trial design, it is quite likely that some participants will not follow the protocol until the outcome data are collected. An important question is then what data to collect for participants who stop the assigned treatment. Sponsors and investigators may believe that the participants are no longer relevant to the study and so be reluctant to incur the costs of continued data collection. Yet continued data collection may inform statistical methods based on assumptions concerning the outcomes that participants might have had if they continued treatment. Continued data collection also allows exploration of whether the assigned therapy affects the efficacy of subsequent therapies (e.g., by improving the degree of tolerance to the treatment through exposure to a similar treatment, i.e., cross-resistance).

The correct decision on continued data collection depends on the selected estimand and study design. For example, if the primary estimand does not require the collection of the outcome after participants discontinue assigned treatment, such as with the estimand (4) above (area under the outcome curve during tolerated treatment), then the benefits of collecting additional outcome data after the primary outcome is reached needs to be weighed against the costs and potential drawbacks of the collection.

An additional advantage of data collection after subjects have switched to other treatments (or otherwise violated the protocol) is the ability to monitor side effects that occur after discontinuation of treatment. Although the cause of such side effects may be unclear (e.g., if a subject switches to another treatment), these data, when combined with long-term followup of other subjects in high-quality epidemiological studies, may help to determine treatment-associated risks that are not immediately apparent. We are convinced that in the large majority of settings, as has been argued by Lavori (1992) and Rubin (1992), the benefits of collecting outcomes after subjects have discontinued treatment outweigh the costs.

Recommendation 3: Trial sponsors should continue to collect information on key outcomes on participants who discontinue their protocol-specified intervention in the course of the study, except in those cases for which a compelling cost-benefit analysis argues otherwise, and this information should be recorded and used in the analysis.

Recommendation 4: The trial design team should consider whether participants who discontinue the protocol intervention should have access to and be encouraged to use specific alternative treatments. Such treatments should be specified in the study protocol.

Recommendation 5: Data collection and information about all relevant treatments and key covariates should be recorded for all initial study participants, whether or not participants received the intervention specified in the protocol.

In EMA’s “Guideline on Missing Data in Confirmatory Clinical Trials”, data collection for dropouts is also recommended.

It should be the aim of those conducting clinical trials to achieve complete capture of all data from all patients, including those who discontinue from treatment.

When patients drop out of a trial, full reporting of all reasons for their discontinuation should be given where possible. This should allow identification of the most important reasons that caused them to discontinue and may influence how these subjects are treated in the missing data analysis. Any follow-up information collected post dropout could be helpful in justifying how these patients are handled in the analyses.

From the implementation standpoint, FDA’s policy regarding this is reflected in its “Guidance for Sponsors, Clinical Investigators, and IRBs Data Retention When Subjects Withdraw from FDA-Regulated Clinical Trials”. The guidance discussed the data collection prior to the withdrawal and after the withdrawal when subjects withdraw from clinical trials.

According to FDA regulations, when a subject withdraws from a study, the data collected on the subject to the point of withdrawal remains part of the study database and may not be removed.

An investigator may ask a subject who is withdrawing whether the subject wishes to provide continued follow-up and further data collection subsequent to their withdrawal from the interventional portion of the study. Under this circumstance, the discussion with the subject would distinguish between study-related interventions and continued follow-up of associated clinical outcome information, such as medical course or laboratory results obtained through non-invasive chart review, and address the maintenance of privacy and confidentiality of the subject’s information.

If a subject withdraws from the interventional portion of the study, but agrees to continued follow-up of associated clinical outcome information as described in the previous bullet, the investigator must obtain the subject’s informed consent for this limited participation in the study (assuming such a situation was not described in the original informed consent form). In accordance with FDA regulations, IRB approval of informed consent documents would be required.

If a subject withdraws from the interventional portion of a study and does not consent to continued follow-up of associated clinical outcome information, the investigator must not access for purposes related to the study the subject’s medical record or other confidential records requiring the subject’s consent. However, an investigator may review study data related to the subject collected prior to the subject’s withdrawal from the study, and may consult public records, such as those establishing survival status.

In a paper by O’Neill and Temple “The Prevention and Treatment of Missing Data in Clinical Trials: An FDA Perspective on the Importance of Dealing With It”, they stated that the data collection after subject withdrawal has its benefits in outcome studies (i.e., the studies with morbidity or mortality events), may not be appropriate for trials with symptomatic measures (usually the quantitative measurement).

A classic remedy, at least in outcome studies, is to attempt to measure outcomes in all the subjects who were initially randomized, including those who withdraw from therapy; this is the “intent to treat” (ITT) approach to the analysis of clinical trial data. As an example of why this might be important, consider an outcome study (with an end point of survival) in which the test drug exacerbated heart failure. In these circumstances, subjects with heart failure, who might be at an increased risk for death, would be more likely to leave the test-drug group. This would lower the mortality risk in the test-drug group and give that drug an advantage with respect to its safety profile, unless the dropouts were followed and the post-dropout events counted. The ITT approach is intended to protect against this kind of “informative censoring” by requiring that dropouts be followed up and that post-dropout events be counted. It is recognized that an ITT analysis is conservative (after all, the benefits of a drug usually disappear once it is stopped), but this is generally considered acceptable in outcome studies. There are compromise approaches—e.g., counting events that occur within 30 days of stopping treatment, assuming that subjects are followed for that duration and it is possible to ascertain outcomes.

Trials of symptomatic benefit generally measure the effects of assigned treatment at successive visits over the duration of the trial, but they typically use the value measured at the final visit as the primary end point. In such trials, the missing-data problem is of a different kind. Early dropouts can leave treatment groups unbalanced with respect to important prognostic patient characteristics related to time-dependent response to treatments. The effect of these dropouts on outcome could go in either direction, i.e., exaggerating or minimizing drug–placebo differences, depending on the reasons for dropping out and whether there were spontaneous (i.e., not drug-related) changes in the outcomes in the study population.

In symptomatic settings, it is not the usual practice to continue to assess effectiveness in patients after they have stopped taking the assigned treatment (ITT approach), as the drug’s effect is assumed to be lost; also, in many cases, an alternative drug is started, and this could influence the outcome for a subject. It is also possible that if a serious adverse end point occurs after the subject has withdrawn from the assigned treatment that event is not captured in the study data. There is also generally less concern, in the symptomatic setting about not capturing a serious study endpoint that was about to occur.

 Another issue is the statistical analysis for the data collected after subjects discontinued the study drug or withdraw from the study. Dr Lisa LaVange had a presentation about “Missing data issues in regulatory clinical trials”. She rightly pointed out the issue post NRC report about the missing data:

Issues encountered in regulatory reviews since publication of the NRC report:-          Some increase in plans to collect data following early discontinuations (of treatment and / or study)
-          But often without plans for how to use the post-discontinuation data collected

Even though the data is collected after the subject withdraw from the study, the data may be used only in sensitivity analysis in an exploratory way, not used in the primary analysis of the study. For example, in a paper “Macitentan and Morbidity and Mortality in Pulmonary Arterial Hypertension”, the vital status for subjects who discontinued from the study was collected, however the mortality analysis with the data post subject discontinuation was only used as an exploratory analysis. It is no surprise that there was no difference between two treatment groups since the placebo patient received the active drug after they discontinued from the study and the treatment effect (if any) will be neutralized. In response to the letter to editor, the author stated the potential biases in such analysis.

With respect to death up to the end of the study, patients who were initially randomly assigned to placebo may have received open-label macitentan after a nonfatal event; this introduced a bias in favor of placebo.

In a book by Domanski and Mckinlay “Successful Randomized Trials: A Handbook for 21 Century”, the utilization of the data collected after subject's withdrawal was discussed. 

Caveat Collection of data, at least on morbidity and mortality, following discontinuation of treatment, does not preclude doing analyses in which these data are not used after treatment is stopped, for example, a nonintent to treat analysis. It is important that at least one analysis be by intent to treat, and if the data are not collected, the analysis is not possible.

In summary, to minimize the missing data in clinical trials, it will be a good practice and a safe approach to continue the data collection after subjects discontinue from the study, especially for outcome studies. The data collected after subjects' withdrawal can be used for performing sensitivity analysis, usually not as the primary analysis approach.  

References:

Strategiesfor dealing with Missing data in clinical trials: From design to Analysis