Clinical Trials Using Historical Control in Rare Disease Drug Development

While the randomized, control clinical trial has become and remains to be the golden standard in drug development, we also see the increased use of non-randomized, single-arm study where the effectiveness of the testing drug is compared to the historical control.

When a pivotal study is a single arm and has no concurrent control, the results from the study will need to be compared to the historical control or a common standard that has been accepted by the medical community or regulatory agencies. This seems to be more common in the oncology area and in rare disease areas.

Without any specific statistics, I can only say this is my impression that the historical control seems to be more accepted by the US FDA in its approvals in oncology and rare disease areas. Here are three examples of recent drug approvals based on historical control:

Venetoclax in Relapsed / Refractory Chronic Lymphocytic Leukemia (CLL)

The pivotal study is a single-arm study without concurrent control and the primary efficacy endpoint is objective response rate (ORR). The result is compared with the historical / standard rate of 40%.

• FDA approves new drug for chronic lymphocytic leukemia in patients with a specific chromosomal abnormality
• FDA NDA medical / statistical reviews
• Stilgenbauer et al (2016) Venetoclax in relapsed or refractory chronic lymphocytic leukaemia with 17p deletion: a multicentre, open-label, phase 2 study. Lancet Oncology

eteplirsen in Duchenne Muscular Dystrophy (DMD)

The original study was a randomized, double-blind, placebo control study with three arms (eteplirsen 30 mg/kg weekly; eteplirsen 50 mg/kg weekly, and placebo) – 4 subjects in each arm for a total 12 subjects. All subjects including placebo subjects were rolled over to an open-label extension study for long-term assessment.

The results from double-blind portion of the study did not provide the strong evidence for efficacy. The sponsor conducted a post-hoc comparison with a historical control.

FDA was not convinced with eteplirsen’s efficacy and conducted the advisory committee. In the end, the eteplirsen was approved as the first treatment in Duchenne Muscular Dystrophy with a lot of controversies. The comparison with historical control (while hotly debated) was a big part of the evidence contributing to the approval.

• FDA grants accelerated approval to first drug for Duchenne muscular dystrophy
• Mendell (2016) Longitudinal Effect of Eteplirsen versus Historical Control on Ambulation in Duchenne Muscular Dystrophy. ANN NEUROL
• FDA NDA Medical Review
• FDA NDA Statistical Review
• 2016 Meeting Materials, Peripheral and Central Nervous System Drugs Advisory Committee

Brineura for Batten Disease

The entire clinical program included: 

• A natural history study with 69 subjects (42 evaluable)
• A Phase 1/2 FIM single-arm study with 24 subjects (23 completed)
• A long-term follow-up study with 23 subjects

Natural history study was based on registry data; Provided the basis as the historical control group

Comparability between natural history study and phase 1/2 study were extensively debated during the review process.

FDA finally approved Brineura for Treating Batten disease in 2017.

• FDA approves first treatment for a form of Batten disease
• A Phase 1/2 Open-Label Dose-Escalation Study to Evaluate Safety, Tolerability, Pharmacokinetics, and Efficacy of Intracerebroventricular BMN 190 in Patients With Late-Infantile Neuronal Ceroid Lipofuscinosis (CLN2) Disease
• FDA NDA Summary Reviews
• FDA NDA Statistical Review

The use of historical control is not new. It was stated in the ICH guideline E10 “CHOICE OF CONTROL GROUP AND RELATED ISSUES IN CLINICAL TRIALS”

Historical control was again mentioned in FDA’s guidance for industry “Rare Diseases: Common Issues in Drug Development”. FDA encourages the natural history study to establish the historical control.

During the FDA advisory committee meeting, Dr Temple gave a presentation about "Historically Controlled Trials": see FDA's presentation slides (Dr Temple's presentation started from page 20).

In FDA's statistical review of eteplirsen in DMD, the following comments were made on the use of historical control: 

Historical Control Cohort
The comparison of eteplirsen with historical controls was not part of an adequate and wellcontrolled study. The applicant obtained historical data after observations were made for the eteplirsen patients. Historical data were obtained from 2 DMD patient registries (Italian DMD Registry and the Leuven Neuromuscular Reference Center – NMRC) for comparison to eteplirsen-treated patients
……
According to the ICH E10 guidance on Control Group and Related Issues in Clinical Trials, the major and well-recognized limitation of externally controlled (including historical control) trials is inability to control bias. The best group and control group can be dissimilar with respect to a wide range of observable and unobservable factors that could affect outcome. It may be possible to match the historical control group to the test group in observed factors but there is no assurance for any unobserved factors. “The lack of randomization and blinding, and the resultant problems with lack of assurance of comparability of test group and control group, make the possibility of substantial bias inherent in this design and impossible to quantitate.”
 Because of the serious concern about the inability to control bias, the use of the external control design is restricted only to unusual circumstances.

• ICH E10 states that “an externally controlled trial should generally be considered only when prior belief in the superiority of the test therapy to all available alternatives is so strong that alternative designs appear unacceptable…” However, such prior belief does not exist for eteplirsen.
• ICH E10 states that “use of external controls should be limited to cases in which the endpoints are objective…” however, performance on the 5-minute walk test can be influenced by motivation. Patients may not achieve maximal 6MWT due to concerns of falling or injury, or patients could try harder with encouragement and with the expectation that the drug might be effective.
• Pocock’s criteria for acceptability of a historical control group require that “the methods of treatment evaluation must be the same,” and “the previous study must have been performed in the same organization with largely the same clinical investigators.” This is especially important when assessing endpoints such as 6MWT, in contrast to hard endpoints such as mortality. For this NDA, these requirements are not met.

Moreover, the historical control group was identified post-hoc in this NDA, leading to potential selection bias that cannot be quantified. If a historical control is to be utilized, selection of the control group and matching on selection criteria should be prospectively planned without knowing the outcome of the drug group and control group.
 Based on ICH E10, “a consequence of the recognized inability to control bias is that the potential persuasiveness of findings from externally controlled trials depends on obtaining much more extreme levels of statistical significance and much larger estimated differences between treatments than would be considered necessary in concurrently controlled trials.” The success criteria for this historical control study were not discussed or pre-specified in the protocol.
 Given all these concerns, including issues of comparability of eteplirsen-treated patients and historical control cohort patients, the fact that 6MWT is not a “hard” efficacy endpoint, the potential of selection bias due to the post-hoc identification of the control cohort by the applicant, and all the known pitfalls with the use of historical controls, the comparison of the eteplirsen with historical control is not statistically interpretable. 

However, even though the statistical review on the use of historical control was very negative, the eteplirsen was still approved as the first drug treating the DMD and presumably the results from the comparison to historical control played the pivotal role in decision. 

In general, the use of historical control can be accepted in some situations especially in rare disease area where there is no approved drug available. Whenever the historical control is used, the following factors need to be considered:

• If the proposed historical control cohort is a priori or post hoc. It is encouraged to collect historical control information from natural history studies. 
• if the patient population from historical control is comparable
• If the outcome measurement is comparable
• if the outcome measurement is a hard endpoint (such as death) or soft endpoint (such as 6MWD)
• If the endpoint measure is easily affected by other factors
• if the endpoint is a soft endpoint (such as objective response rate ORR), whether or not any approach is implemented to avoid the bias (such as using central reader)

Generating graph / figure in publication quality

As I was recently preparing a poster for ATS Internal Conference, I was told that the plots I provided were not in high quality. When placing the plots on the poster, they became blurry. I realized that the issue was with the DPI. 

DPI is used to describe the resolution number of dots per inch in a digital print and the printing resolution of a hard copy print dot gain. High DPI = High Resolution.

The journal may have a requirement for the minimum resolution for the graphs and figures, for example, the PLOT One requires the resolution in the range of DPI 300-600. The Science magazine has the following requirement: 

Resolution. For manuscripts in the revision stage, adequate figure resolution is essential to a high-quality print and online rendering of your paper. Raster line art should have a minimum resolution of 600 dots per inch (dpi) and, preferably, should have a resolution of 1200 dpi. Grayscale and color artwork should have a minimum resolution of 400 dpi, and a higher resolution if possible.

Wiley had a paper discussing the challenges the authors might face for providing the high resolution figures. See the editorial: How to meet dots per inch requirements for images

I used SAS procedure sgplot to create the plots. The default DPI is 100, which is too low for publication or poster. Fortunately, there are easy ways to change the DPI for the output plots. Below are some programs for doing so: 

*default DPI=600; low resolution;
ods listing gpath='c:Temp\';
ods graphics on;
proc sgplot data=sashelp.stocks (where=(date >= "01jan2000"d
                                 and date <= "01jan2001"d
                                 and stock = "IBM"));
   title "Stock Volume vs. Close";
   vbar date / response=volume;
   vline date / response=close y2axis;
run;
ods graphics off;
ods listing close;

*set DPI=400;
ods listing gpath="c:\Temp\" dpi=400;
ods graphics on;
proc sgplot data=sashelp.stocks (where=(date >= "01jan2000"d
                                 and date <= "01jan2001"d
                                 and stock = "IBM"));
   title "Stock Volume vs. Close";
   vbar date / response=volume;
   vline date / response=close y2axis;
run;
ods graphics off;
ods listing close; 

*set DPI =400 and also use style=journal;
ods listing gpath="c:\Temp\" style=journal dpi=400;
ods graphics on;
proc sgplot data=sashelp.stocks (where=(date >= "01jan2000"d
                                 and date <= "01jan2001"d
                                 and stock = "IBM"));
   title "Stock Volume vs. Close";
   vbar date / response=volume;
   vline date / response=close y2axis;
run;
ods graphics off;

ods listing close;

When high DPI is chosen, the size of the file will increase. For the same plot, the file sizes for three plots (in png format) above are 26, 160, and 158 kb. 

The following program will service the same purpose, but use ods pdf commend. 

ods pdf file="c:\Temp\test.pdf";

proc sgplot data=sashelp.stocks (where=(date >= "01jan2000"d

                                 and date <= "01jan2001"d

                                 and stock = "IBM"));

   title "Stock Volume vs. Close";

   vbar date / response=volume;

   vline date / response=close y2axis;

run;

ods pdf close;

ods pdf file="c:\temp\test.pdf" dpi=600;

proc sgplot data=sashelp.stocks (where=(date >= "01jan2000"d

                                 and date <= "01jan2001"d

                                 and stock = "IBM"));

   title "Stock Volume vs. Close";

   vbar date / response=volume;

   vline date / response=close y2axis;

run;

ods pdf close;

ods pdf file="c:\Temp\test.pdf" style=journal dpi=600;

proc sgplot data=sashelp.stocks (where=(date >= "01jan2000"d

                                 and date <= "01jan2001"d

                                 and stock = "IBM"));

   title "Stock Volume vs. Close";

   vbar date / response=volume;

   vline date / response=close y2axis;

run;

ods pdf close;

The same issue with graph quality is also true when we use R. There is a RES = option to select the desired DPI. Please see the blog post by Daniel Hocking "High Resolution Figures in R".