FDA Guidance "Development and Licensure of Vaccines to Prevent COVID-19" - sample size situation for phase 3 studies

Last week, FDA issued its guidance for industry "Development and Licensure of Vaccines to Prevent COVID-19". Unlike the usual FDA guidance where FDA issues the draft guidance with a comment period, this guidance is immediately effective as the final version upon its issuance.

The guidance sets its expectations for the development and licensure of vaccines to prevent coronavirus disease (COVID-19), including considerations for manufacturing, nonclinical and clinical studies, and post-licensure requirements. Also, Dr. Peter Marks, director of the Center for Biologics Evaluation and Research, shed light on the reasoning behind the agency’s 50% efficacy threshold and where the agency stands on challenge trials and emergency use authorizations (EUAs). See the article "Marks on COVID-19 vaccine efficacy, EUAs and challenge trials"

The guidance provided details about the pivotal (phase 3) efficacy and safety study. For phase 3 study, the primary efficacy endpoint should be "the incidence of laboratory-confirmed symptomatic COVID-19" specified in the guidance as the following: 

Section E of the guidance 'Statistical Considerations' provided the specific requirements for the statistical success criteria (i.e., point estimate of vaccine efficacy at least 50% and the lower bound of the alpha-adjusted confidence interval at least 30%). 

What does the vaccine efficacy of 50% mean?

Vaccine Efficacy (VE) = [(COVID-19 attack rate in the unvaccinated group - COVID-19 attack rate in the vaccinated group) / COVID-19 attack rate in the unvaccinated group] * 100%

where the attack rate is equivalent to the incidence rate. 

Using the relative risk (RR) or risk ratio [= (incidence of COVID-19 cases in the vaccinated group) / (incidence of COVID-19 cases in the unvaccinated group)], VE = 1 - RR.

Suppose after 3-6 months observation period post-vaccination, there are 100 cases of laboratory-confirmed symptomatic COVID-19 patients in the unvaccinated group and 50 cases in the vaccinated group, assuming the total follow-up time (person-time) are similar between the unvaccinated and vaccinated groups, the VE will be calculated as (100-50)/100 *100%= 50%.

In practice, the person time (PT) in the vaccinated group and unvaccinated group will be included in the calculation of attack rate or incidence rate where the attack rate = the number of cases observed in the vaccination group or unvaccinated group / Person Time (PT) in the vaccination or vaccination group. If the Poisson regression method is used, the person time will be used in the model as an offset variable. Person time (PT) is the same concept as person-year or patient-year and can be calculated in the same way as the person year with a perhaps different unit. 

VE at least 50% is a point estimate - not dependent on the sample size. For a much smaller trial, if we have 5 cases in the vaccination group and 10 cases in the vaccination group, the VE will still be 50%.

For sample size calculation, we will also need to know the confidence interval. As indicated in the FDA's guidance,  the lower bound of the appropriately alpha-adjusted confidence interval around the primary efficacy endpoint point estimate needs to be greater than 30%.

How to calculate the sample size? Which parameters do we need to calculate the sample size? 

The commercial software (such as EAST, SAS Proc Power, PASS, NQuery Advisor) can all be used to calculate the sample size. In a hypothetic example in the previous post, the sample size was calculated using EAST module for Ratio of Poisson Rates. 

Matthew M. Loiacono had a thesis on "Sample Size Estimation and Power Calculations for Vaccine Efficacy Trials for Exceedingly Rare Diseases" where two SAS macros were developed to estimate the sample sizes: one based on the Exact Conditional Test and one based on the Z Test Normal Approximation.

The sample size calculation will need the following five parameters:

• Incidence of laboratory-confirmed symptomatic COVID-19 cases in the unvaccinated group 
• True efficacy of test vaccine under the alternative hypothesis (according to FDA guidance, this is 50%) 
• Minimum efficacy of test vaccine, under the null hypothesis (according to FDA guidance, this is 30%)
• Power: pre-specified statistical power desired to achieve (usually 80% or 90%) 
• Alpha: pre-specified maximum one-sided level of the test (usually 0.05 for experimental level)

The most critical parameter is the incidence of laboratory-confirmed symptomatic COVID-19 cases - it is difficult to predict; it shifts with geographic location and time; it is impacted by the COVID-19 prevention strategies and policies. In general, the lower the incidence of COVID-19 cases, the larger the sample size is needed for phase 3 study to demonstrate the vaccine efficacy. 

Assuming the incidence of COVID-19 is 0.01 in unvaccinated (placebo) group, with 80% statistical power and alpha = 0.05, using the SAS macro based on Exact Conditional Test method, 33868 volunteers need to be randomized (estimated 254 COVID-19 cases observed) to detect the vaccine efficacy with point estimate at least 50% and lower bound of 95% confidence interval greater than 30%.