Safety Assessment Committee (SAC), Data Monitoring Committee (DMC), Steering Committee (SC), Clinical Event Committee (CEC)

In clinical trials, there are usually different committees established to ensure the study protocol is executable, to ensure the integrity of the study, to monitor the safety of the clinical trial participants. The most common one is the Data Monitoring Committee (DMC) - previously called Data Safety Monitoring Board (DSMB), which is critical for conducting the interim analysis for safety or for efficacy (whether the interim results are too good (overwhelming efficacy) or futility (unlikely to achieve the statistical significance if the trial continues)). In FDA's Guidance for Clinical Trial Sponsors "Establishment and Operation of Clinical Trial Data MonitoringCommittees", DMC was defined as the following: 

A clinical trial DMC is a group of individuals with pertinent expertise that reviews on a regular basis accumulating data from one or more ongoing clinical trials. The DMC advises the sponsor regarding the continuing safety of trial subjects and those yet to be recruited to the trial, as well as the continuing validity and scientific merit of the trial. When a single DMC is responsible for monitoring multiple trials, the considerations for establishment and operation of the DMC are generally similar to those for a DMC monitoring a single trial, but the logistics may be more complex. For example, multiple conflict of interest determinations may be needed for each DMC member.

In this same guidance, two other clinical trial committees were mentioned to be compared with the DMC: Steering Committee (SC) and Endpoint Adjudication Committee (EAC). EAC may be called Clinical Endpoint Committee (CEC). 

Clinical Trial Steering Committees In some clinical trials the sponsor may choose to appoint a steering committee; this committee may include investigators, other experts not otherwise involved in the trial, and, usually, representatives of the sponsor. A sponsor may delegate to a steering committee the primary responsibility for designing the study, maintaining the quality of Contains Nonbinding Recommendations 7 study conduct, ongoing monitoring of individual toxicities and adverse events, and, in many cases, writing study publications. When there is a steering committee, the sponsor may elect to have the DMC communicate with this committee rather than directly with the sponsor. Interactions between the steering committee and the DMC consist primarily of discussions during "open sessions" (see Section 4.3) of DMC meetings and the communication of recommendations following each DMC review of the trial. More extensive interactions might occur when early termination is being considered, or when external forces (e.g., announcement of results of related studies) impact the ongoing trial.

Endpoint Assessment/Adjudication Committees Sponsors may also choose to establish an endpoint assessment/adjudication committee (these may also be known as clinical events committees) in certain trials to review important endpoints reported by trial investigators to determine whether the endpoints meet protocol-specified criteria. Information reviewed on each presumptive endpoint may include laboratory, pathology and/or imaging data, autopsy reports, physical descriptions, and any other data deemed relevant. These committees are typically masked to the assigned study arm when performing their assessments regardless of whether the trial itself is conducted in a blinded manner. Such committees are particularly valuable when endpoints are subjective and/or require the application of a complex definition, and when the intervention is not delivered in a blinded fashion. Although such committees do not share responsibility with DMCs for evaluating interim comparisons, their assessments (if performed at frequent intervals throughout the trial with results incorporated into the database in a timely manner) help to ensure that the data reviewed by DMCs are as accurate and free of bias as possible.

In FDA guidance for Industry "Safety Assessment for IND Safety ReportingGuidance for Industry", a separate committee, Safety Assessment Committee or SAC, in short, was proposed. The SAC was defined as 

"... a group of individuals chosen by the sponsor to review safety information in a development program and tasked with making a recommendation to the sponsor regarding whether the safety information must be reported in an IND safety report"

In a separate Guidance for Industry, "Sponsor Responsibilities— Safety Reporting Requirements and Safety Assessment for IND and Bioavailability/Bioequivalence Studies", the term 'Safety Assessment Committee" was not used, but the responsibility of a 'safety assessment team' was discussed. 

All of these committees can be established to ensure patient safety and study integrity. The responsibilities across these committees are sometimes overlapped. The table below is created to compare these committees side by side:

	Data Monitoring Committee (DMC)	Steering Committee (SC)	Clinical Event Committee (CEC)	Safety Assessment Committee (SAC)
Alternative names	Data Safety Monitoring Board (DSMB) Data Safety Monitoring Committee (DSMC)		Event Assessment Committee (EAC) Event Adjudication Committee (EAC)
Responsibilities	Monitoring for effectiveness, safety, study conduct; Executing the adaptation rules for adaptive designs; making recommendations for study modification, pause, termination	Designing the study, maintaining the quality of study conduct, ongoing monitoring of individual toxicities and adverse events, and, Helping the sponsor to execute the study (for example enrollment); Writing study publications.	Reviewing important endpoints reported by trial investigators to determine whether the endpoints meet protocol-specified criteria	Reviewing safety information in a development program; Performing aggregate safety analyses; and making recommendations regarding whether the safety information must be reported in an IND safety report
Participating in study design and protocol development	May provide the comments on the protocols	Yes	May provide inputs related to the clinical events to be adjudicated	Limited to the safety reporting section of the study protocol
External	Usually external	External	External	Usually internal within the sponsor
Independent	Usually independent	Usually not    independent	Independent	Independent of the study team
Review aggregate data	Safety and maybe also Efficacy	Secondary review	Usually not	Safety data only
Blinding	Unblinded to the treatment assignments. DMC can review fully unblinded data	Blinded	Usually blinded	Blinded
Participating in clinical trial result reporting/publication	Usually not	Yes	Usually not	Usually not
Documents	DMC Charter DMC Statistical Analysis Plan	SC Charter is optional	CEC Charter, Operational Manual, May employ a computer system to facilitate the adjudication	Safety Surveillance Plan

Human Preclinical Studies and Phase 0 Clinical Trials

The drug development process includes various steps from discovery (discovery of a new compound or biological product) to preclinical research (measuring the safety, toxicity, and efficacy in animal models), and then to clinical trials (testing the safety and efficacy in humans - either healthy volunteers or patients). The clinical trials are phased from phase 1 & 2 (early phase trials) to phase 3 (pivotal, confirmatory, late-phase trials), and to phase 4 (post-marketing clinical trials). A diagram indicates various stages of drug development. With innovative clinical trial designs, the clinical trial phases are blurred, for example, seamless phase 1/2 trial and seamless 2/3 trial. In rare disease areas, the drug development process may not have all phases of clinical trials. The adequate and well-controlled study may be phase 3, phase 2, or phase 1 (for example, expansion cohort studies).

Preclinical development, also called preclinical study or nonclinical study, is a stage of research that begins before clinical trials (testing in humans) and during which important feasibility, iterative testing and drug safety data are collected, typically in laboratory animals. Traditionally, the phase 1 study may be the first-in-human trials with the purpose of studying the drug's:

• pharmacokinetics (ADME: absorption, distribution, metabolism, and excretion) and pharmacodynamics (enzyme, protein,...) 
• toxicity, safety and side effects associated with increasing doses
• maximum tolerable dose
• early evidence of effectiveness

There are now two new steps that may be utilized in drug development: Preclinical human studies and phase 0 clinical trials. A revised drug development diagram is as follows: 

Human pre-clinical studies are those pre-clinical studies utilizing the human subjects (either utilizing the specimens collected from human subjects or performing testing in human decedents). Human pre-clinical studies are 'pre-clinical' because they are not conducted under IND (investigational new drug) and they are conducted for collecting the data to support the IND-enabling studies. 

In a paper by Abdallah et al, "A novel prostate cancer immunotherapy using prostate-specific antigen peptides and Candida skin test reagent as an adjuvant", they described a human pre-clinical study where peptides based on the prostate-specific antigen amino acid sequences were evaluated in terms of their recognition by peripheral immune cells from prostate cancer patients using interferon-γ enzyme-linked immunospot assay. A sample size of 10 patients with prostate cancer was selected for the study. The authors concluded: 

"We described a human preclinical study of a novel prostate cancer immunotherapy consisting of PSA peptides and Candida skin test reagent as an adjuvant. As solubility and formulation have been developed, it would be feasible to further evaluate the utility of this new therapy particularly when a proportion of prostate cancer patients seem to have immune cells with the ability to recognize these PSA peptides already. Therefore, whether this immunotherapy may enhance immune responses to PSA leading to tumor regression should be examined."

Dr. Locke's team in UAB recently conducted pioneer xenotransplantation of a gene-modified pig kidney. The study was described in the paper by Porrett et al "First clinical-grade porcine kidney xenotransplant using a human decedent model". The pig kidney was transplanted to a human decedent (brain dead patient). The purposes of this human preclinical study were stated as the following:

"Xenotransplantation is arguably the most pragmatic solution to the organ shortage crisis, but safety and efficacy concerns have limited advancement into humans. In preparation for a phase I clinical trial of porcine renal xenotransplantation at the University of Alabama at Birmingham, we asked what gaps in knowledge must be filled before such a clinical trial could be ethically offered to research subjects. We thus aimed to develop a human preclinical model which would permit the in vivo evaluation of critical safety and feasibility tenets of the pig-to-NHP model without risk to a living human. Our study was designed to test five central questions: (1) Is the current suite of porcine genetic modifications sufficient to avoid hyperacute rejection in humans? (2) Would prospective flow-based crossmatching correlate with graft survival free of hyperacute rejection? (3) Would life-threatening intraoperative complications occur during a renal porcine xenotransplant? (4) Would porcine cells and/or pathogens be detected in the blood of a human recipient? (5) Could porcine renal xenotransplantation be safely performed under the conditions necessary for a clinical trial? To this end, we designed and performed this experiment under clinical-grade conditions which included the transplantation of 10-GE porcine kidneys designed specifically for human transplantation into the conventional anatomic position using processes and facilities in compliance with multiple regulatory agencies."

In human preclinical studies, while human subjects are involved, there is no IND needed. Consents by the patients (in the first example) or by the relatives (in the second example) are needed. 

Phase 0 Clinical Trial: the concept of phase 0 clinical trial came from the FDA's guidance for industry "Exploratory IND Studies". The term 'phase 0 clinical trial' was not used in the guidance, but was used for exploratory IND studies. Phase 0 clinical trial may now be called 'early phase 1' clinical trial, for example, in NIH.gov website and in clinicaltrials.gov:

According to the FDA guidance "Exploratory IND Studies", the exploratory IND studies (therefore phase 0 clinical trials or early phase 1 clinical trials) are defined as the following:

"Exploratory IND studies usually involve very limited human exposure and have no therapeutic or diagnostic intent. Such studies can serve a number of useful goals. For example, an exploratory IND study can help sponsors

• Determine whether a mechanism of action defined in experimental systems can also be observed in humans (e.g., a binding property or inhibition of an enzyme)
• Provide important information on pharmacokinetics (PK)
• Select the most promising lead product from a group of candidates5 designed to interact with a particular therapeutic target in humans, based on PK or pharmacodynamic (PD) properties
• Explore a product’s biodistribution characteristics using various imaging technologies

 Whatever the goal of the study, exploratory IND studies can help identify, early in the process, promising candidates for continued development and eliminate those lacking promise. As a result, exploratory IND studies may help reduce the number of human subjects and resources, including the amount of candidate product, needed to identify promising drugs. The studies discussed in this guidance involve dosing a limited number of subjects with a limited range of doses for a limited period of time.

Existing regulations provide more flexibility with regard to the preclinical testing requirements for exploratory IND studies than for traditional IND studies. However, sponsors submitting the kinds of studies described in this guidance have not always taken full advantage of that flexibility. Sponsors often provide more supporting information in their INDs than is required by the regulations. Because exploratory IND studies involve administering either subpharmacologic doses of a product, or doses expected to produce a pharmacologic, but not a toxic, effect, the potential risk to human subjects is less than for a traditional phase 1 study that, for example, seeks to establish a maximally tolerated dose. Because exploratory IND studies present fewer potential risks than do traditional phase 1 studies that look for dose-limiting toxicities, such limited exploratory IND investigations in humans can be initiated with less, or different, preclinical support than is required for traditional IND studies.  "

In cancer.org website,"Types and Phases of Clinical Trials", Phase 0 clinical trials were specifically mentioned: 

Phase 0 clinical trials: Exploring if and how a new drug may work

Even though phase 0 studies are done in humans, this type of study isn’t like the other phases of clinical trials. The purpose of this phase is to help speed up and streamline the drug approval process. Phase 0 studies may help researchers find out if the drugs do what they’re expected to do. This may help save time and money that would have been spent on later phase trials.

Phase 0 studies use only a few small doses of a new drug in a few people. They might test whether the drug reaches the tumor, how the drug acts in the human body, and how cancer cells in the human body respond to the drug. People in these studies might need extra tests such as biopsies, scans, and blood samples as part of the process.

Unlike other phases of clinical trials, there’s almost no chance the people in phase 0 trials will benefit. The benefit will be for other people in the future. And because drug doses are low, there’s also less risk to those in the trial.

Phase 0 studies aren’t widely used, and there are some drugs for which they wouldn’t be helpful. Phase 0 studies are very small, often with fewer than 15 people, and the drug is given only for a short time. They’re not a required part of testing a new drug.

Phase 0 clinical trials are mainly conducted in the oncology area and there are quite some phase 0 (or early phase 1) studies are listed in clinicaltrials.gov. An example of a phase 0 study was published in JCO (Kummar et al 2009 "Phase 0 Clinical Trial of the Poly (ADP-Ribose) Polymerase Inhibitor ABT-888 in Patients With Advanced Malignancies").

FDA's Priority Review Voucher for COVID-19 Vaccine

 To spur drug development in the under-developed area, FDA relies on various policies and programs to incentify the biotech and pharmaceutical companies. The most famous one is the orphan drug development. In the last 10 years or so, a program called 'priority review voucher' become popular. If the biotech and pharmaceutical companies develop a drug in certain underdeveloped areas, once the drug is approved (not emergency user authorization), FDA will issue the marketing authorization holder a priority review voucher. The priority review voucher can be redeemed for future NDA/BLA applications to shorten the review time by four months. The priority review voucher can be a commodity to be sold or transferred to another company for monetary gain. 

• FDA's Priority Review Voucher Programs

• RAPS: Regulatory Explainer: Everything You Need to Know About FDA’s Priority Review Vouchers
• Endpoints: Priority review voucher updates: Who's won PRVs, who's bought them, and how much they're selling for

FDA currently has three priority review voucher programs and each has its own guidance to industries:

• Tropical Disease Priority Review Voucher Program and its guidance
• Rare Pediatric Disease Priority Review Vouchers
• Material Threat Medical Countermeasure Priority Review Vouchers 

To qualify for a PRV for tropical disease, a sponsor’s application must be for a drug or biological product for the prevention or treatment of a “tropical disease,” where a list of tropical diseases are listed by FDA and additional tropical diseases may be added by FDA. For example, Ebola was not in the original list but was added in 2019.

To qualify for a PRV for rare pediatric disease, a sponsor's application must be for a drug or biological product for the prevention or treatment of a "rare pediatric disease," where the rare pediatric disease is defined as:

The 21st Century CuresAct (Cures Act) Section 565A of the FD&C Act was designed to encourage development of new drug and biological medical countermeasures (MCMs), by offering additional incentives for obtaining FDA approval of certain MCMs. While there are existing incentive programs to encourage the development and study of drugs and biologics that may also be applicable to MCMs, section 565A of the FD&C Act provides an incentive specifically for development of certain MCMs, which may be used alone or in some cases in combination with other incentive programs. Other FDA incentive programs include: orphan-drug designation and the associated benefits under the Orphan Drug Act for rare disease drugs; programs to encourage study of drugs used in pediatric populations; various programs to facilitate and expedite development and review of new drugs to address unmet medical needs in the treatment of serious or life-threatening conditions; and programs for certain tropical disease products and antibacterial products. 

Last week (Jan 31, 2022), Moderna obtained the final approval (not emergency user authorization) for its Covid-19. After the approval, a brand drug name or proprietary name - SPIKEVAX - is approved. In the approval letter, FDA granted Moderna a material threat medical countermeasure priority review voucher (PRV).

Endpoints had an article "Exclusive: The curious case of the BioNTech priority review voucher" indicating that for Pfizer/BioNTech's Covid-19 vaccine (brand name Comirnaty) approval, the priority review voucher was not included at the time of the BLA approval. After the approval, BioNTech (as the market authorization holder) received a MCM priority review voucher even though the company did not announce it publicly. 

A priority review voucher can be worth hundreds of millions if transferred. however, the monetary awards from the MCM priority review voucher are trivial compared to the sales of the Covid-19 vaccines - PRV is just a nice-to-have incentive.  

Randomization, Re-Randomization, and Micro-Randomization

 I recently saw a Twitter post mentioning "Micro-Randomized Study Design Example - Maryland Alcohol-Dependent Moms Abstinence (MAMA) Study" and found the term 'micro-randomization' interesting and prompted me to compare the concept of randomization, re-randomization, and micro-randomization. Based on the number of times that a subject can be randomized in a study, we can differentiate the studies as randomized, re-randomized, and micro-randomized trials. 

Randomization is the process of assigning subjects (patients, clinical trial participants) by chance to groups that receive different treatments. In the simplest trial design (parallel-group design), the investigational group receives the new treatment and the control group receives standard therapy. At several points during and at the end of the clinical trial, researchers compare the groups to see which treatment is more effective or has fewer side effects. Randomization helps prevent bias. Bias occurs when a trial's results are affected by human choices or other factors not related to the treatment being tested.

ICH Topic E 9Statistical Principles for Clinical Trials has an entire section discussing randomization as the key design technique to avoid biases:

"2.3.2 Randomisation

Randomisation introduces a deliberate element of chance into the assignment of treatments to subjects in a clinical trial. During subsequent analysis of the trial data, it provides a sound statistical basis for the quantitative evaluation of the evidence relating to treatment effects. It also tends to produce treatment groups in which the distributions of prognostic factors, known and unknown, are similar. In combination with blinding, randomisation helps to avoid possible bias in the selection and allocation of subjects arising from the predictability of treatment assignments.

The randomisation schedule of a clinical trial documents the random allocation of treatments to subjects. In the simplest situation it is a sequential list of treatments (or treatment sequences in a crossover trial) or corresponding codes by subject number. The logistics of some trials, such as those with a screening phase, may make matters more complicated, but the unique pre-planned assignment of treatment, or treatment sequence, to subject should be clear. Different trial designs will require different procedures for generating randomisation schedules. The randomisation schedule should be reproducible (if the need arises).  

......"

In typical clinical trials, the study participants will be randomized only one time whether to different treatments or different treatment sequences. For clinical trials with parallel-group design, subjects are randomized to receive one of two or more treatments. For clinical trials with cross-over design, subjects are randomized to follow one of two or more treatment sequences. Once the treatment sequence is determined, subjects will follow the sequence to receive multiple treatments (for example, treatment A then treatment B or treatment B than treatment A,...)

The vast majority of randomized clinical trials are falling into this category and this includes:

• randomized double-blind trials: randomization + blinding 
• randomized open-label trials: randomization without blinding
• randomized cross-over trials: randomization to the sequence of treatments
• adaptive randomized trials: adjust the randomization ratio
• "N of 1" clinical trials: can be considered as a high order crossover, once the sequence is decided, the treatments at various stages are decided

Re-randomization is the process describing a situation where each patient can be randomized more than one time in the same study.  There are two types of re-randomization:

re-randomization in SMART trial design framework - SMART stands for Sequential Multiple Assignment Randomized Trial. In a trial with SMART designs, the same subject may be randomized more than once depending on the response to the initial assigned treatment after the initial randomization. According to the paper by Kidwell et al "Sequential, Multiple Assignment, Randomized Trial Designs in Immuno-oncology Research", A SMART is a multistage, randomized trial in which each stage corresponds to an important treatment decision point. Participants are enrolled in a SMART and followed throughout the trial, but each participant may be randomized more than once. Subsequent randomizations allow for unbiased comparisons of post-initial randomization treatments and comparisons of treatment pathways. The goal of a SMART is to develop and find evidence of effective treatment pathways that mimic clinical practice.

In a review paper by Wallace at el "SMART Thinking: a Review of Recent Developments in Sequential Multiple Assignment Randomized Trials", the following general diagram was given for SMART design:

We saw that SMART design with re-randomization was used in clinical trials in different therapeutic areas:

In a paper by Almirall et al "Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research", the following diagram was used to illustrate a SMART design for weight loss research. After the initial randomized treatment period, the responders and non-responders are identified. The non-responders were re-randomized to different treatments. 

Ruppert et al described a study with SMART design in CLL "Application of a sequential multiple assignment randomized trial (SMART) design in older patients with chronic lymphocytic leukemia" where patients with complete response after stage 1 were re-randomized to receive two different treatments at stage 2. 

We conducted an ICE study - a registration study with IGIV-C in CIDP (a rare neurology disease) "Intravenous immune globulin (10% caprylate chromatography purified) for the treatment of chronic inflammatory demyelinating polyradiculoneuropathy(ICE study): a randomised placebo-controlled trial". We did not explicitly state the SMART design but did employ the re-randomization in the study. The subjects who were responders (to the blinded treatment) were re-randomized to receive either IGIV-C or Placebo in additional six months follow-up period. The re-randomized portion of the study was to compare the relapse rate between two treatment groups - a key secondary efficacy endpoint. 

With the re-randomized portion of the study, we built in two randomizations in the same study and demonstrated the treatment effect of IGIV-C in the primary efficacy endpoint of improving the responder rate and also the treatment effect of IGIV-C in preventing the relapse in the additional follow-up period - essentially two studies in one. This was used as a rationale for a single pivotal trial (two studies in one) to provide substantial efficacy for effectiveness. 

Re-randomization is also discussed for use in different settings where the subjects who complete the initial randomized period are put back to the randomization pool. Subjects were re-randomized to the study as if they are new to the study. In other words, the same subject was re-used and re-randomized into the study. Kahan et al described this type of re-randomized trial as the following: 

This type of re-randomized design is very rarely used and may be used in clinical trials with ultra-rare diseases that patient recruitment is extremely challenging. 

A Micro-randomized trial (MRTs) can be considered as an extension of the SMART design. The same subject can be randomized and re-randomized many times to different interventions. The time scale is much more frequent and short (for example several times in a day). The term 'micro-' can be confusing, but it is used to differentiate this type of randomization from the classical setting where randomization can not be too frequently. The term 'micro-' is used to describe a setting where the randomization/re-randomization needs to be conducted more frequently on a much short time scale - almost continuous time points. A Micro-randomized trial is good for the interventions that are delivered through mobile devices (such as push notification) and is good for interventions that are intended for changing subjects' behaviors. 

Here is a website describing what the micro-randomized trial is:

In micro-randomized trials (MRTs), individuals are randomized hundreds or thousands of times over the course of the study. The goal of these trials is to optimize mobile health interventions by assessing the relative effect of different intervention options and assessing whether the intervention effects vary with time or the individual's current context. With MRTs we can gather data to construct optimized just-in-time adaptive interventions (JITAIs).

Intervention options can include either or both engagement strategies and therapeutic treatments. Consider the Heartsteps MRT (described below) that is designed to promote physical activity among sedentary people. Heartsteps includes phone notifications with tailored activity suggestions to encourage physical activity; these are therapeutic in focus. On the other hand the SARA MRT (also described below) is designed to promote engagement by young adults in substance abuse research. SARA includes rewards for participants who complete assessments; these are engagement strategies. The design of both of these projects can be seen in the “Projects Using MRTs” section, below.

In an MRT, each participant can be randomized many times. For example in the Heartsteps project, the researchers identified five times throughout the day when people are mostly likely to be available to take a brief walk. At each of the five time points, the application randomizes between delivering a phone notification containing a tailored activity suggesion or to not deliver anything; as a result over the course of the 42 days, each participant is randomized 210 times. This sequence of both within-participant and between-participant randomizations comprises the MRT.

The MRT data can then be used to assess the effectiveness of the tailored activity suggestions and to build rules for when to deliver the suggestions in order to help individuals be more active. To do this the application records a variety of outcomes. In this case, the app collects the minute-by-minute step count from the participant’s activity-tracking wristband throughout the day, the participant’s overall level of physical activity, and the participant’s context at each of the 5 times per day (using GPS to determine the person’s location and the local weather). The resulting data is used by researchers to assess the effectiveness of the activity suggestions and to build rules for when and where to deliver the suggestions. In other MRTs, the randomization could apply to what type of intervention to provide, rather than whether or not to provide an intervention. The ultimate goal of Heartsteps is the development of a JITAI that will successfully encourage higher levels of physical activity. The study design of the MRT used in Heartsteps is shown below.

MRTs are an emergent innovation in behavioral science.

We are in the digital era and digital tools will become more used in interventions (especially the adaptive intervention) for lifestyle and behavior changes. However, we don't think that the 'micro-randomized trials' will be suitable for drug trials for registration purposes. 

• On this date, there are 16 registries in clinicaltrials.gov using micro-randomized trials. Not surprisely, all micro-randomized trials are in the behavioral science field.  
• Xu et al "Multi-Level Micro-Randomized Trial: Detecting the Proximal Effect of Messages on physical activity"
• “Micro-randomized Trials in Mobile Health” with Zhenke Wu, PhD
• Dr. Susan Murphy: Micro-randomized Trials for JITAI Development

Paired T-test and McNemar's test for paired data based on the summary data

Sometimes, it is necessary for us to calculate the p-values based on the summary (aggregate) data without the individual subject level data. In a previous post, group t-test or Chi-square test based on the summary data was discussed. Group t-test and chi-square test can be used in the setting of parallel-group comparisons. 

In single-arm clinical trials, there is no concurrent control group, and the statistical test is usually based on the pre-post comparison. For continuous measures, the pre-post comparison can be tested using paired t-test based on the change from baseline values (i.e., post-baseline measures - baseline measures):  For discrete outcomes, the pre-post comparison may be tested using McNemar's test.

Paired t-test:

A paired t-test is used when we are interested in the difference between two variables for the same subject. Suppose we have the descriptive statistics for change from baseline values: 83 subjects had the outcome measures at both baseline and week 12 (therefore, 83 pairs), the mean and standard deviation for these 83 pairs are: 10.7 (70.7); 68 subjects had the outcome measures at both baseline and week 24 (therefore 68 pairs), the mean and standard deviation for these 68 pairs are 20.2 (80.9). 

With the mean difference, the standard deviation for differences, and the sample size (# of pairs), we have all the elements to calculate the t statistics and therefore the p-value using the formula below. 

This can be implemented in SAS as the following - t-statistics and p-values can be calculated for each of weeks 12 and 24 based on the aggregate data. 
 

McNemar's Test:

McNemar's test is a statistical test used on paired nominal data. It is applied to 2 × 2 contingency tables with a dichotomous trait, with matched pairs of subjects, to determine whether the row and column marginal frequencies are equal (that is, whether there is "marginal homogeneity"). In clinical trials, the aggregate data may not be obvious as a 2 × 2 contingency table but can be converted into a 2 × 2 contingency table. 

Suppose we have the following summary data for post-baseline week 12: the number and percentage of subjects with improvement, stable (no change), and deterioration categories. 

		All subjects (n=300)
Week 12	Improved	54 (18%)
	No Change	228 (76%)
	Deteriorated	18 (  6%)

At Week 12, there are more subjects in the 'Improved' category than in the 'Deteriorated' category even though the majority of subjects are in the 'No Change' category. Are they more subjects with improvement than deterioration? 

Assuming that change from category 1 to 0 is 'Improved' and change from category 0 to 1 is 'Deteriorated', the table above can be converted into a 2 × 2 table: 

		Baseline
		0	1
Week 12	0	228	54
	1	18	0

or

		Baseline
		0	1
Week 12	0	0	54
	1	18	228

For McNemar’s test, only the numbers in the diagonal discordant cells (in our case, the # of improved and the # of deteriorated) are relevant.

The concordant cells (in our case, the # of no change) will only contribute to the sample size (therefore the degree of freedom), not have an impact on the p-value. How the # of subjects with the ‘No Change’ is split doesn’t matter with our calculation of chi-square statistics and therefore the p-value.

For the data highlighted in yellow, McNemar’s test can be performed using the SAS codes like this (weight statement indicates count variable is the frequency of the observation and agree option requests McNemar's test). How the 228 subjects in the concordance ‘No Change’ category are split has no impact on the p-value calculation. 

Overrunning issue at the interim analysis for group sequential design

It is pretty common these days that clinical trials (especially the late phase, adequate, and well-controlled studies) employ interim analyses to determine if the efficacy results are too good so that the study should be stopped early for overwhelming efficacy, or if the efficacy results are not good so that the study should be stopped early for futility, or both. A study with formal interim analyses to look at the comparative efficacy is called 'group sequential design' even though the 'group sequential design' may not be formally used in the study protocol. Group sequential design is the most common type of adaptive design as described in the FDA guidance "adaptive designs for clinical trials". 

As mentioned in an early post "overrunning issues in adaptive design clinical trials", one of the issues with interim analyses in group sequential design is the overrunning issue. Overrunning consists of extra data, collected by investigators while awaiting results of the interim analysis (IA). Overrunning is the 
phenomenon that data will continue to accumulate after it is decided to stop a trial (Whitehead, 1992). In many cases there will be patients who have already been admitted to the trial but whose responses are not yet known. Also some extra patients will enter the trial because of the delay between the moment the data for the final interim analysis were retrieved, and the moment participating clinical centers receive instruction to stop recruitment.

EMA guidances "reflection paper on methodological issues in confirmatory clinical trials planned with an adaptive design" had a full paragraph discussing the overrunning issue: 

When planning for an interim analysis, the decision needs to be made about the timing of the interim analysis and what data is to be included in the interim analysis. For an event-driven study where the number of events is the endpoint, the timing of the interim analysis can be based on the percentage of the events, for example, the interim analysis can be performed when 50% of the total number of events are accrued. In other words, once 50% of the total number of events is achieved. For a longitudinal design, the endpoint is measured at various intervals. At any time during the study, there will be patients at different stages of the study (reaching the end of the study, reaching a specific duration of the study, or just being randomized into the study). It is more difficult to determine a good timing for interim analysis. Suppose that an interim analysis is planned when 50% of subjects reach the end of the study, by that time, there will be plenty of subjects in the various stage of the study already, just having not reached the end of study yet. If the study enrollment is pretty fast and the study endpoint is pretty long (for example 52 weeks), by the time 50% of subjects reach the study endpoint, the majority of subjects (if not all) have already been randomized into the study. 

If the interim analysis results trigger the recommendation of discontinuing the study early (either for efficacy or futility), the debate is whether the interim analysis needs to be re-run by including the overrunning subjects before adopting the recommendation to discontinue the study early. 

This exact issue about the handling of the overrunning subjects was discussed in Biogen's aducanumab clinical trials in Alzheimer's disease. As I discussed in a previous post "Futility Analysis and Conditional Power When Two Phase 3 Studies are Simultaneously Conducted", Biogen made the wrong decision and discontinued its pivotal studies (EMERGE and ENGAGE) where one of the studies was later found to have statistically significant treatment effects. The wrong decision was driven by two issues: 

• they calculated the conditional powers based on the pooled data from both studies (instead of calculating the conditional powers separately based on the data from individual studies) - this was discussed in the previous post. 
• they stopped the study early without re-running the interim analysis by including the overrunning subjects. 

The overrunning issue was mentioned in a recent article in the Wall Street Journal (Jan 4, 2022) "How Biogen Fumbled Its Alzheimer's Drug ---Once-promising Aduhelm is pricey and without proven efficacy"

"By evaluating data midstream in approval-seeking trials, companies can try to predict whether a drug will succeed if the trial continues. Stopping trials early for "futility," in industry parlance, can save millions of dollars and prevent patients from investing hope on an ineffective drug.

In a March 2019 meeting, Biogen executives on a small "senior decision team," as the company called it, concluded that the trials were doomed. Biogen pulled the plug and asked researchers around the world to shut down trials. It told more than 3,000 Alzheimer's patients who had volunteered that they would no longer receive treatment.

Biogen stock fell by nearly 30% the day of the announcement.

Biogen executives made errors in shutting down the trials. The trial plan called for analyzing data after half of patients completed the study treatment in late December 2018. By the time Biogen completed the analysis in March 2019, three more months of additional data were available -- but the decision team didn't scrutinize the additional data before the company halted the trials, Biogen has said.

A Biogen consultant in the summer of 2018 recommended to senior Biogen statisticians that they consider all available trial data, according to a person involved in the process. The consultant cautioned them that a plan to leave out consideration of additional trial data after the cutoff date -- and to leave out certain data from patients in the trial before the cutoff -- would open up Biogen to criticism and scrutiny, the person said. The statisticians didn't heed the consultant's advice, and it isn't clear whether the decision team or management considered the recommendation, the person said.

Biogen declined to comment on past discussions with its consultants but said it followed its pre-established statistical-analysis plan.

The decision not to consider the three months of additional data was a misstep, said some clinical-trial experts and statisticians. "Additional data after a study stops is called 'overrunning.' We plan for it," said Scott Emerson, a professor emeritus of biostatistics at the University of Washington who served on an FDA advisory committee that recommended against approving Aduhelm in November 2020. "In this case, the overrunning data was large."

The Biogen spokeswoman said: "Our decision to stop the trials, though clearly incorrect in hindsight, was based on putting patients at the forefront -- as it always should be. Cost was not considered in determining futility."

Only in the weeks after the trials stopped did Biogen scientists complete a preliminary analysis of the overrunning data and recognize their mistake, the Biogen spokeswoman said. The data seemed to show that one of the trials would have produced positive results, despite the likelihood of a negative outcome in the second trial. Initially, Biogen had analyzed combined data from both trials."

Even though the aducanumab was finally approved by FDA through the accelerated approval pathway, the approval was very controversial - the first drug and the only drug so far that was approved based on studies that had been stopped early for futility. There are strong pushback from academic about the use of aducanumab because of its unapproved efficacy and perhaps also because of the drama of resurrecting a drug that was declared failed. 

We all wonder: had these two pivotal studies not been stopped for futility by the sponsor, what would be the situation now for aducanumab?