Rule of three

Rule of three states that consider a Bernoulli random variable with unknown probability p, if in n independent trials no events occur, a quick-and-ready approximation to the upper 95% confidence bound for p is 3/n.
This rule has particularly been used in pre-licensure clinical trials where the adverse event rate is very rare. Sample sizes of pivotal trials for licensure are set for an efficacy endpoint, and vary according to the indication. Therefore, pivotal confirmatory studies provide adequate denominators for determining adverse events that occur at a frequency higher than or similar to the clinical efficacy outcome. However, sample sizes are not sufficient for detecting the rare events. Only reliable post-marketing surveillance systems will allow detection of a rare adverse event or a small increase in adverse event rate.
Rule of three provides a quick calculation of the upper confidence interval of the observed rate (observed rate is zero when there is no event occurred). It is based on the estimated upper limit of the 95% confidence interval when this particular event has not occurred during a clinical trial or during the clinical development program. As an example in vaccine development program, if no event has been observed with a sample size of 100, the upper limit of the 95% CI of the rate of this event is 3%. A sample size in the range of 10 000 subjects can be considered adequate for establishing the protective efficacy of a new vaccine. If no event of any particular sort has been observed during a pre-licensure clinical program involving 10 000 individuals, it can be estimated that this event rate has an upper limit of 3 per 10 000. Rarer adverse events, those occurring at a lower frequency than the vaccine-targeted disease, or an increase in rare adverse events, are unlikely to be detected before licensure their assessment must rely on post-marketing studies (phase IV).
However, if the rare event does occur during the clinical trial (the event rate is not zero), the rule of three should not be used. Instead, the confidence interval should be calculated according to exact or permutation approach (not the formula that is based on the normal approximation).

Also, we should not attempt to calculate the sample size based on the rule of three. The sample size should still be based on the efficacy endpoint instead of the comparison of the rare events – otherwise, the sample size could be huge.

References:
1. Ernst Eypasch et al. Probability of adverse events that have not yet occurred: a statistical reminder. BMJ 1995
2. Steve Simon's web blog. Stistical confidence interval with zero event