Sunday, January 26, 2014

Overrunning Issue In Adaptive Design Clinical Trials

Adaptive designs have been more and more commonly used from early phase clinical trials to late phase pivotal clinical trials and from the limited therapeutic areas (such as oncology) to broad therapeutic areas. While adaptive design can have different forms (group sequential design,  seamless phase II/III design, sample size re-estimation, futility design,…), it all requires formal interim analyses with results to be reviewed by the independent Data Monitoring Committee. The rule for adaptation is pre-specified and the decision for adaptation is based on the interim analysis results.

In the phased clinical trials, the period between the analysis of phase II data and recruitment of phase III patients, is called “white space”. During the “white space” period, the results from early phase studies are digested and the additional time is needed for the study start up including regulatory submissions. The common perception is that with adaptive design such as seamless phase II/III design, the ‘white space’ can be eliminated – that is why the term ‘seamless’ is used in the first place. However, with adaptive design, the ‘white space’ may be shortened, not be eliminated. In studies with adaptive designs, there is usually no break in patient enrollment between the phases or stages or no break while preparing for the interim analysis. This brings in another issue – overrunning issue.

During the period while waiting for the endpoint data available for interim analysis, If the treatment duration is too long, too many patients would be randomized during the transition period – so called ‘overrunning’, which could result in inefficiencies and losing the benefits of the adaptive design. 

Recruitment rate relative to treatment data availability is the critical factor for overrunning. According to a paper by Judith Quinlan and Michael Krams (Implementing adaptive designs: logistical and operational considerations, the ideal ratio of recruitment / treatment duration is 4.

“As a rule of thumb we propose to establish whether the overall recruitment duration is at least four times the observational period required before the primary endpoint reads out in any one patient.  For example, in a stroke trial where each patient is observed over a period of 3 months, an adaptive design could be considered if the trial is open for recruitment for 12 months or longer.  This view may need to be modified, should there be a good early predictor of final outcome, allowing for the deployment of a longitudinal model.  For example, in acute stroke it might be possible to use early measurements of the stroke scale to predict final outcome (Grieve and Krams, 2005).  Should the early observation be a good predictor of the final outcome, we may consider using it in our assessment of weighing recruitment speed versus time needed before endpoint readout.  To formally establish the “optimal” recruitment speed, we propose to conduct clinical trial simulations, mimicking the potential real life environment of the trial and exploring the impact of longitudinal models.”

The overrunning issue was not discussed in FDA’s guidanceAdaptive Design Clinical Trials for Drugs and Biologicals”, however, it is discussed in EMA guidanceReflection Paper In Methodological Issues in Confirmatory Clinical Trials Planned With an Adaptive Design”. Section 4.1.3 of this guidance has specific discussions about the overrunning issue. 

"4.1.3 Overrunning
In many clinical trials the primary endpoint is not observed immediately for each patient (e.g. survival or time to event data). Furthermore, in trials with a complex organisational structure, additional patients are likely to be randomised or some even followed to the primary endpoint before the results of a pre-planned interim analysis are known. If a trial is to be terminated as a result of an interim analysis it is always important to carry out an additional analysis including all of these further patients that did not contribute to the interim analysis. It may be that when this analysis is carried out, the null hypothesis can no longer be rejected and apparently decision making may depend on whether or not these so called overrunning patients are included or excluded from the analysis. In such a situation, it is accepted regulatory practice to base decision making on the final results of the trial (not the interim analysis). This is also in accordance with the intention to treat principle that all randomised patients should be analysed. Obviously, overrunning patients need to be treated and observed according to the protocol and due attention should be given to this at the planning stage of the trial.

A full discussion of the results of a trial should be based on estimates of the treatment effect rather  than simply on P-values alone. If the estimate of the treatment effect including the overrunning patients is not very different from that excluding them, then a small increase in the P-value might not be regarded as a concern. An important reduction in the size of the point estimate might, in contrast, lead to reluctance to accept the overall result as “positive”, especially as, unless a trial is stopped very early, the proportion of overrunning patients will usually be sufficiently small such that the estimate of the treatment effect should not be substantially altered. In all cases, results including and excluding the overrunning patients should be presented and differences between these two analyses should be discussed.”

The overrunning issue is discussed in many adaptive clinical trial implementations. Here are some of them:

No comments: