Estimate the number of subjects needed to detect at least one event and the rule of three

It is sometimes important to know how likely a study is to detect a relatively uncommon event, particularly if that event is severe, such as intracranial hemorrhage (ICH), progressive multifocal leukoencephalopathy (PML), bone marrow failure, or life-threatening arrhythmia. A great many people must be observed in order to have a good chance to detecting even one such event, much less to establish a relatively stable estimate of its frequency. For most clinical trials, sample size is planned to be sufficient to detect main effects in efficacy and sample size is likely to be well short of the number needed to detect these rare events. Sometimes, we do see the request from FDA for a sample size large enough to detect at least one rare event.

Cempra is a pharmaceutical company in Chapel Hill, North Carolina with a very promising antibiotics drug. However, their NDA submission was not approved by FDA, not due to the efficacy, but due to the safety issue (precisely, due to FDA’s concern about the potential liver toxicity). In FDA’s complete response letter (CRL, in other words, rejection letter), it says:

“To address this deficiency, the FDA is recommending a comparative study to evaluate the safety of solithromycin in patients with CABP. Specifically, the CRL recommends that Cempra consider a study of approximately 9,000 patients exposed to solithromycin to enable exclusion of serious drug induced liver injury (DILI) events occurring at a rate of approximately 1:3000 with a 95 percent probability.”

The request for a study with these many subjects is unusual for a pre-market study. It will certainly not be feasible for an antibacterial drug in community-acquired bacterial pneumonia indication. It is interesting to see that the FDA applied the ‘rule of three’ in proposing the sample size.

The rule of three says: to have a good chance of detecting a 1/x events, one must observe 3x people. For example, to detect at least one event if the underlying rate is 1/1,000, one would need to observe 3,000 people. In Solithromycin case, to detect at least one DILI event if the background rate of DILI event is 1/3,000, the number of subjects to be observed would be 3 x 3,000 = 9,000 subjects.

I had previously written an articleabout the rule of three from the angle of calculating the 95% confidence interval when there is no event occurred in x number of subjects. The same rule can be applied to the situation for estimating the sample size to detect at least one event.    

In one of our studies with a thrombolytic agent, FDA is concerned about the potential side effect of intracranial hemorrhage (ICH). In FDA’s comment on IND, FDA asked us to have a safety database containing at least 560 subjects exposed to the tested thrombolytic agent. They stated:

In order to permit the occurrence of 1 ICH event without necessitating halting the study, the safety database would have to have at least 560 subjects exposed to your XYZ drug with only one ICH event to keep the ICH rate under 1%. Please comment.

In this case, the background rate of ICH is assumed to be 1%. Based on the rule of three, the safety database would be 3 x 100 = 300. However, the number of 560 is coming from the direct calculation assuming a binomial distribution.

Based on an ICH rate of 1%, assuming a binomial distribution, to detect at least one ICH event, 560 subjects will be needed in order to keep upper bound of the exact two-sided 95% confidence interval below 1%.

A small SAS program below can be used for the calculation. Adjust the sample size and then see what the exact 95% confidence intervals are.

 data test;

  input ICH $ count;

  datalines;

  Have 1

  No   560

run;

proc freq data=stopping;

  weight count;

  tables ICH /  binomial (p=0.01) alpha=0.05 cl;

           *p=0.01 indicates the background rate;

  exact binomial;  *Obtain the exact p-value;

run;

In clinical studies with potential side effect of rare events, if we need to base the safety database on detecting at least one such event, the sample size could be very large. As we see from the calculation or simply apply the rule of three, the sample size largely depends on the assumed background event rate. It is usually the case that the background event rate is from the literature. The literature usually have variety of the results. Using DILI as an example, the background rate from the literature could vary depending on if we talk about the rate in normal population, in community-acquired bacterial pneumonia patients, or in community-acquired bacterial pneumonia patients treated with other antibiotics. 

REFERENCES:

• Confidence interval with zero event
• Wikipedia: Rule of Three (statistics)
• Zero Defect Sampling
• Gathering Adequate Clinical Trial Data for the Postmarketing Safety Database