Sunday, June 28, 2020

Sample Size for COVID-19 Vaccine Phase 3 Study

Vaccine development for COVID-19 is at historic speed. Several companies have already had vaccines in the early phase of clinical trials. Leading the pack are AstraZeneca / Oxford University and Moderna / NIH who are targeting July for starting the late-stage phase 3 studies. Phase 3 study is pivotal in demonstrating that the vaccine is safe and effective in the general population including those at high risk of contracting COVID-19 and those with underly diseases.

There are quite some discussions about the sample size of the phase 3 study - the sample size for phase 3 study should be adequate to provide sufficient statistical power to demonstrate the vaccine's efficacy in preventing the COVID-19 infection and also adequate for regulatory agencies to assess the safety of the vaccine - sample size should be big enough so that the rare event (if any) can be observed. The sample size is being proposed to be at least 30,000 volunteers.
For pivotal clinical trials, the sample sizes depend on the study design and the primary efficacy endpoint; and estimated based on assumptions.

The study design will be traditionally randomized, double-blinded, placebo- or active-controlled, parallel-group design (not human challenge design even though it has been pushed by some people) - see a previous post "Human Challenge Study Design for Covid-19 Vaccine Clinical Trials?".
The primary efficacy endpoint is the incidence of symptomatic COVID-19. Moderna has finalized the study design for its phase 3 study and it says that the study will include 30,000 volunteers and the primary and secondary efficacy endpoints as the following:
"The primary objective of the trial, which is set to start in July, is to assess the ability of mRNA-1273 to prevent symptomatic COVID-19 disease. Secondary endpoints will assess the ability of mRNA-1273 to prevent hospitalization and infection with SARS-CoV-2."
AstraZeneca/Oxford's COVID-19 vaccine enters phase 2/3 clinical trial. Their proposed sample size is much smaller than 30,000 subjects mentioned by US experts. We can also notice that they propose to use the vaccine against meningococcal bacteria as the control (instead of placebo). 
"Researchers at the University of Oxford have begun enrolling subjects in a phase 2/3 clinical trial of AstraZeneca-partnered COVID-19 vaccine AZD1222. The next stage of the program, which follows a 1,000-subject phase 1, is set to enroll 10,260 people in the U.K. to generate results to support the first shipments to customers in September."
"Once the vaccine moves into phase 3, the researchers will limit enrollment to people age 18 years and older. Adult participants in the phase 2 and 3 trials will be randomized to receive one or two doses of AZD1222 or a vaccine against meningococcal bacteria that will serve as the control.

The use of an active vaccine as a control is intended to ensure participants are unable to tell whether they received AZD1222 based on side effects such as soreness at the injection site. In the absence of such effects across both groups, participants could determine whether they had received the vaccine and make behavioral changes that skew the results of the study. "
Baseline on the study registration on clinicaltrials.gov "A Phase 2/3 Study to Determine the Efficacy, Safety and Immunogenicity of the Candidate Coronavirus Disease (COVID-19) Vaccine ChAdOx1 nCoV-19", we can see the following:
  • The study consisted of different sub-groups (different age groups and different duration of the follow-up period).
  • The primary efficacy endpoint is the number of biologically confirmed (PCR positive) symptomatic cases of COVID-19
  • The primary safety endpoint is the occurrence of serious adverse events (SAEs) throughout the study duration.
It is not clear what the assumptions are used for estimating the sample size (30,000 subjects for Moderna's phase 3 study or 10,260 subjects for AstraZeneca/Oxford's phase 3 study). Perhaps the sample size is more based on the experiences rather than the calculation based on the solid assumptions.
In order to estimate the sample size, we will need to know the incidence rate of symptomatic COVID-19 in the control group (subjects who receive a placebo or a none anti-COVID-19 vaccine) and then the effect size (how much reduction in the incidence rate of symptomatic COVID-19 in the vaccine group (subjects who receive COVID-19 vaccination) - both of these are difficult to obtain.

The incidence rate of symptomatic COVID-19 can vary significantly depending on the timing, the location, and the prevention and COVID-19 control strategies (i.e., stay-at-home, quarantine, social distance).

Back in January and February, the incidence rate of symptomatic COVID-19 would be very high. But now the COVID-19 situation in China has been under control. It is no longer feasible to test the efficacy of the COVID-19 vaccine in China. There are reports that COVID-19 vaccine researchers in China are looking for foreign sites to recruit the volunteers to test the efficacy of their vaccine candidate.

In the US, the incidence rate of symptomatic COVID-19 would be very high in New York and New Jersey in March/April time, now the states with high incidence rates have shifted to the southern states such as Texas, Arizona, North Carolina.

For phase 3 study to demonstrate the efficacy against the COVID-19, a sufficient number of subjects need to be included in the study so that (hopefully) a sufficient number of symptomatic COVID-19 cases can be observed (more in the control group and less in COVID-19 vaccine group). If subjects are recruited in areas with a high incidence rate, it will be quicker to accrue the number of symptomatic COVID-19 cases. Ironically, with strict COVID-19 prevention/control strategies, by the time the phase 3 studies start, the situation may be under the control and the incidence rate may be too low to accrue enough symptomatic COVID-19 cases. In the article below, a drop in coronavirus cases was listed as one of the biggest risks for Moderna's phase 3 study.

"3. A drop in coronavirus cases

Of course, a drop in coronavirus cases is great news for everyone. But in order for Moderna and other vaccine makers to test their investigational products, the virus must be actively circulating. In an outbreak situation, the vaccine or placebo is administered to a group of healthy volunteers. If a high number of placebo participants get sick and those who received the vaccine don't, it's likely the vaccine is working. But if the virus is hardly present, it would be impossible to draw such a conclusion.

In the U.S., coronavirus cases recorded by the Centers for Disease Control and Prevention are cumulative, so they will continue to grow. And some states are still seeing spikes. But nationally, the number of people seeking medical attention for symptoms has been on the decline. If the trend continues, it may present difficulties for Moderna and rivals conducting trials in the U.S. to prove vaccine efficacy.

That doesn't mean the vaccine is doomed, but testing it could become more complicated. If virus circulation drops considerably, researchers may conduct a "challenge" trial. That means healthy volunteers are immunized, then exposed to the virus. There also is the possibility of conducting trials in other areas where the virus is on the rise. At this point, Moderna hasn't said it would turn to either of these alternatives."
The primary efficacy endpoint of "biologically confirmed symptomatic cases of COVID-19" belongs to the count data. Given the incidence rate is very low among the general population, the data can be assumed to follow a Poisson distribution or negative binomial distribution. The sample size calculation will then need to be based on Poisson rates or negative binomial rates. 

Hypothetically, the statements about the sample size for phase 3 COVID-19 vaccine studies can be something like this assuming 50% reduction in symptomatic CIVID-19 cases in the vaccine group:
Assuming that the incidence rate (Poisson mean) of subjects with symptomatic COVID-19 case is 0.003 (0.3%) in control group and 0.0015 (0.15%) in COVID-19 vaccination group, 32,673 subjects needs to be randomized to have 80% statistical power to detect the treatment difference."
In the US, we saw the rise in coronavirus cases - it is bad, really bad, but ironically it is good for vaccine clinical trials - it is easier and quicker to accrue a sufficient number of symptomatic COVID-19 cases and it requires less sample size for phase 3 studies to demonstrate the difference between vaccinated and control groups. As Dr. John Skerritt from the Department of Health, Australia said about the COVID-19 pandemic: "Do not waste a good crisis". 
at

1 comment: