Mediation analysis and SAS CAUSALMED procedure

In a recent publication (Benza et al "Contemporary Risk Scores Predict Clinical Worsening in Pulmonary Arterial Hypertension - An Analysis of FREEDOM-EV"), we conducted an analysis called 'Mediation analysis'. In the statistical analysis section, the 'mediation analysis' was stated as the following: 

"To determine whether the change in Week 12 REVEAL Lite 2 risk score ‘mediated’ the treatment effect in delaying clinical worsening, we used SAS (v14.3) CAUSALMED procedure which operationalizes the work of Valeri and VanderWeele.

This analysis attempts to determine what fraction of the total treatment effect appears to be attributable to the treatment effect on the REVEAL Lite 2 score. The analysis was adjusted for baseline REVEAL Lite 2 score; we did the analysis both with and without assuming that there is a treatment and mediator (REVEAL Lite 2 score) interaction on the outcome model (clinical worsening). The definition ‘net clinical benefit’ has been previously proposed as the achievement of all three French non-invasive low risk factors without a clinical worsening event; we retrospectively used the present database to model the performance of this definition."

According to Wikipedia, the mediation model and mediation analysis are defined as the following: 

In statistics, a mediation model seeks to identify and explain the mechanism or process that underlies an observed relationship between an independent variable and a dependent variable via the inclusion of a third hypothetical variable, known as a mediator variable (also a mediating variable, intermediary variable, or intervening variable). Rather than a direct causal relationship between the independent variable and the dependent variable, a mediation model proposes that the independent variable influences the mediator variable, which in turn influences the dependent variable. Thus, the mediator variable serves to clarify the nature of the relationship between the independent and dependent variables.

Mediation analyses are employed to understand a known relationship by exploring the underlying mechanism or process by which one variable influences another variable through a mediator variable. In particular, mediation analysis can contribute to better understanding the relationship between an independent variable and a dependent variable when these variables do not have an obvious direct connection.

A mediator variable can either account for all or some of the observed relationship between two variables. 

Full Mediation

Maximum evidence for mediation, also called full mediation, would occur if the inclusion of the mediation variable drops the relationship between the independent variable and dependent variable to zero. In other words, the effect of the independent variable on the dependent variable is all through the mediator variable. 

Partial mediation

Partial mediation maintains that the mediating variable accounts for some, but not all, of the relationship between the independent variable and dependent variable. Partial mediation implies that there is no only a significant relationship between the mediator and the dependent variable, but also some direct relationship between the independent and dependent variable - the line from independent variable to dependent variable is solid and c is not equal to zero. 

 

The mediation analysis has been used in the data analysis for observational data, clinical trial data, survey data, and epidemiology study data. 

In an article by Eyre et al "Effect of Covid-19 Vaccination on Transmission of Alpha and Delta Variants", the mediation analysis was used to assess whether the effect of the vaccination status of the index patient was explained by Ct values at diagnosis.  Ct values are cycle-threshold values (indicative of viral load21) in the index patient.

In an article by Reaven et al "Intensive Glucose Control in Patients with Type 2 Diabetes — 15-Year Follow-up", mediation analyses were performed:

In prespecified mediation analyses, Cox proportional-hazards models were used to examine the effects of the glycated hemoglobin level on the primary cardiovascular disease outcome and on the observed treatment effects. Specifically, the log-linear association of the cumulative glycated hemoglobin level (modeled as a time-varying covariate) with the primary cardiovascular disease outcome was assessed during the period of separation of the glycated hemoglobin curves and after convergence. Models examined the effect of treatment group (intensive therapy or standard therapy) on the primary outcome in an unadjusted analysis (model 1) or while accounting for baseline, most recent, or cumulative mean glycated hemoglobin level (models 2, 3, and 4, respectively).

A paper by Vo et al summarized "the conduct and reporting of mediation analysis in recently published randomized controlled trials: results from a methodological systematic review"

Mediation analysis can be performed using SAS procedure CAUSALMED. CAUSALMED procedure was developed for estimating causal mediation effects from observational data, but can definitely be used for estimating mediation effects from the randomized controlled clinical trial data. Please see the references below:

• SAS documentation about CAUSALMED procedure including the application examples
• Yung et al Causal Mediation Analysis with the CAUSALMED Procedure
• Yung Introducing the CAUSALMED procedure for causal mediation analysis - Youtube video 
• Youtube video: Modeling causal effect using SAS/Stat

Mediation analysis can be performed using other software. This is very well summarized in a paper by Valente et al "Causal Mediation Programs in R, Mplus, SAS, SPSS, and Stata".

Handling of values below or above a threshold (Below the Low Limit of Quantification or Above the Upper Limit of Quantification)?

In clinical trials, the samples are often collected and sent to the central laboratory or specialty laboratory for measuring certain parameters (drug concentrations, metabolite concentrations, biomarkers,...). It is not uncommon that the results may be reported as "<xxx" or ">xxx" indicating that the measurement is below or above a threshold, outside the range, or the quality control curves. We call them below the low limit of quantification or above the upper limit of quantification.

How to handle them in the data set and in the analyses?

In the data set, the laboratory results should be reported as it is in character variable. The data listings should use the character variable so that the signs of '<' or '>' will be kept and displayed.

For the purpose of the statistical summaries and analyses, a separate numeric variable should be derived and appropriate rules will be applied to these values below the low limit of quantification or above the upper limit of quantification.

 
Most of the discussions were about the handling of the values below the low limit of quantification (BLQs). See a previous post "BLQs (below limit of quantification) and LLOQ (Lower Limit of Quantification): how to handle them in analyses?" and the researchgate.net discussion board "How should one treat data with <LOQ values during statistical analysis?".

The options for handling the BLQs are:

• Treat BLQs as missing
• Treat BLQs as 0
• Treat BLQs as 1/2 of the LLQ (lower limit of qualification). For example, if the result was reported as "<10" µg, take 5 µg as the measure - this approach is pretty common in handling pharmacokinetic concentration data. 
•  Simply remove the sign of  '<' and take the face value (i.e. LLQ value). For example, if the result was reported as "<10" µg, take 10 µg as the measure. 
• More complicated methods using statistical (regression, maximum likelihood,...) approaches 

There are fewer discussions about handling the values above the upper limit of quantification (ULQ). Usually, these values above the upper threshold will be handled by:

• Treat values above ULQ as missing
• Simply remove the sign of  '>' and take the face value (i.e. ULQ value). For example, if the result was reported as ">100" mg, take 100 mg as the numeric value

In an SAP developed by Astellas, the simple rule was specified for handling the values below or above a threshold:

"For continuous variables that are recorded as “< X” or “> X”, the value of “X” will be used in the calculation of summary statistics. The original values will be used for the listings."

In an SAP developed by Galapagos for their phase 3 study of GLPG1690 in subjects with idiopathic pulmonary fibrosis, the following rules were proposed to handle values below or above a threshold. Their approach of adding or deducting a small number from the face value is unconventional.

7.3. Handling of Values Below (or Above) a Threshold 

Values below (above) the detection limit will be imputed by the value one unit smaller or larger than the detection limit itself. In listings, the original value will be presented. Example: if the database contains the value “<0.04”, then for the descriptive statistics the value “0.03” will be used. The value “>1000” will be imputed by “1001”. 

Placebo effect and its impact on the overall treatment effect

RCTs (randomized, controlled clinical trials) are still the golden standard in clinical research. In RCTs, the most common control group is the Placebo. According to Wikipedia, a placebo is a sham substance or treatment which is designed to have no known therapeutic value. Common placebos include inert tablets (like sugar pills), inert injections (like saline), sham surgery, and other procedures. In order to maintain the blinding (masking), the placebo group may include additional excipients similar to the experimental drug so that the placebo group will have the same characteristics as the experimental drug in shape, size, color, texture, weight, taste, smell,......

Placebo is assumed to have no therapeutic effect or detrimental effect. However, the assumption may not be true especially when the composition of the placebo is in consideration. 

In previous article "Placebo Effect, Honest Placebo, Open-label Placebo", we discussed the placebo effect in diseases in the CNS and psychological area or in diseases with subjective symptom measures. In the post "Placebo effect and the choice of placebo", we discussed the composition of the placebo and some 'placebo' may actually have therapeutic effect. For example, in clinical trials to test the therapeutic effects of IGIV, the  low concentration of albumin may be selected as the placebo control - the low concentration of albumin may actually have the therapeutic effect. In both of these cases, the placebo effect or potential therapeutic effect from the 'placebo' can cause the unexpected higher response rate in Placebo arm, therefore decrease the difference between the experimental drug and the placebo groups, result in the failed trials. 

On the flip side, the placebo can have detrimental effect. There are quite some recent discussions about the placebo having the detrimental effect - consequently, the overall treatment effect observed in the clinical trials may not be due to the therapeutic effect of the experimental drug, but due to the detrimental effect of the placebo group. In other words, the overall treatment effect can be  overestimated due to the detrimental effect of the placebo.

Here are some articles discussing the potential detrimental effects of the placebo in clinical trials to study the effects of fish oil in the prevention of the cardiovascular events. The REDUCT-IT trial was published in NEJM and was the pivotal trial resulting in the FDA and EMA's approval. According to the study protocol, "the matching placebo capsule is filled with light liquid paraffin and contains 0 mg of AMR101 (icosapent ethyl [ethyl-EPA])." The detrimental effect of the placebo may come from the paraffin. 

• Effects of Randomized Treatment With Icosapent Ethyl and a Mineral Oil Comparator on Interleukin-1β, Interleukin-6, C-Reactive Protein, Oxidized Low-Density Lipoprotein Cholesterol, Homocysteine, Lipoprotein(a), and Lipoprotein-Associated Phospholipase A2: A REDUCE-IT Biomarker Substudy
• Amarin stands behind Vascepa after new analysis raises questions about its efficacy
• New analysis of Amarin’s Vascepa resurrects big questions about effectiveness
• New Biomarker Data Add to Concerns Over REDUCE-IT Trial
• Mineral oil: safety and use as placebo in REDUCE-IT and other clinical studies

When we conduct the placebo-controlled clinical trials, the composition of the placebo needs to be carefully considered and the potential therapeutic effect from the placebo needs to be minimized. 

CRO, ARO, and VRO - All types of clinical research organizations

CRO (Contract Research Organization or Clinical Research Organization)

A CRO is a company that provides clinical trial management services for the pharmaceutical, biotech, and medical device industries.

Although there are different types of CROs and diverse levels of specialization (distinct therapeutic areas for instance), typical CRO services include regulatory affairs, site selection and activation, recruitment support, clinical monitoring, data management, trial logistics, pharmacovigilance, biostatistics, medical writing, and project management, among others. Given that the clinical trials are going global, the large CROs usually have foot prints in all of these countries that are the usual spots for doing clinical trials.

In a clinical trial, CROs are hired by sponsors to perform a set of tasks, taking various technical and administrative responsibilities on the sponsor’s behalf.

The main role of the CRO is to plan, coordinate, execute, and supervise the processes involved in the development of a clinical trial, being a central contact point between the sponsor and other trial actors (e.g. ethics committees, regulatory agencies, vendors, and hospitals).

CROs are key players in clinical research, since they have the knowledge and the capabilities needed for the proper development of a clinical study. They help sponsors by reducing their workload, while ensuring trial quality and compliance with national and international standards.

At the same time, many CROs supply innovative technological tools to increase efficiency in the study processes, which translates into cost reductions.

Without doubt, CROs play a crucial role in the success of a clinical trial. Sponsors should carefully assess the particular needs of their projects, and look for the CRO that best meets their technical requirements and budget.

The website https://www.proclinical.com/blogs/2022-3/top-10-cros-to-watch-in-2022 listed the top ten CROs in 2022 which are:

• PPD (now part of ThermoFisher)
• KCR
• ICON (it just acquired PRA International)
• IQVIA (I still like their previous name Quintiles)
• Parexel
• Medpace
• Syneos Health
• Labcorp (it just acquired Covance)
• CTI
• PSI

I would also add WCG as one of the top CROs.

ARO (Academic Research Organization)

An ARO is an academic or non-profit organization that provides support mainly to the academic principal investigators in the form of research services: biopharmaceutical development, biologic assay development, commercialization, preclinical research, clinical research, clinical trials management, and pharmacovigilance.

Comparing to the CRO, AROs are non-profit; are usually cheaper, are usually strictly related to their main center; are mainly focused on studies oriented to improve the standard clinical practice, thus they are majorly involved in observational studies, phase IV studies, pragmatic clinical trials,...

ARO can also be good to take on the role of the central reader and clinical event adjudications. 

This book chapter discussed the role of ARO in clinical research:

Chapter 3. The Role of Academic Research Organizations in Clinical Research

The most prominent ARO is perhaps the Duke Clinical Research Institute (DCRI). 

VRO (Virtual Research Organization)

With the recent rapid development in conducting the virtual clinical trials or decentralized clinical trials, the new term VRO is appearing. VRO is specialized in helping the sponsors to do virtual clinical trials. A paper by Hong et al "Virtual Research Organization: Nature and Forms" discussed the roles of the VROs. 

An example of a VRO is ObvioHealth - they claim to be the VRO who are specialized in doing decentralized clinical trials. 

Adverse event / serious adverse event data entry when death event occurs

If the death event is an efficacy endpoint in the study, should the death event still be reported as AE/SAE? 

If the death (or mortality) is an efficacy endpoint, the death event will ordinarily not be reported as AE/SAE. Only in some special situation, for example, a death due to anaphylactic reaction, car accident,... that are not related to the underlying disease, will the death be reported as AE/SAE. 

FDA's guidance Safety Reporting Requirements for INDs and BA/BE Studies clearly stated this: 

Generally, study endpoints refer to outcomes that sponsors are measuring to evaluate efficacy. For trials designed to evaluate the effect of a drug on disease-related mortality or major morbidity, endpoint information should be collected, tracked, and monitored, usually by a Data Monitoring Committee (DMC), during the course of the study. The protocol would prespecify a monitoring plan for determining whether subjects receiving the drug treatment are at higher risk for the outcome (e.g., all-cause mortality), and such results would be reported according to the protocol. The study endpoints must be reported to FDA by the sponsor according to the protocol, and ordinarily would not be reported as IND safety reports, except when there is evidence suggesting a causal relationship between the drug and the event (21 CFR 312.32(c)(5)). For example, a death ordinarily would not be reported as an individual case in an expedited report from a trial designed to compare all-cause mortality in subjects receiving either drug treatment or a placebo. On the other hand, in the same trial with an all-cause mortality endpoint, if the death occurred as a result of an anaphylactic reaction that coincided with initial exposure to the drug, or as a result of fatal hepatic necrosis, the death must be reported as an individual case in an IND safety report because there would then be evidence suggesting a causal relationship between the drug and the event (21 CFR 312.32(c)(5)).  

In our clinical trial "Dinutuximab and Irinotecan Versus Irinotecan to Treat Subjects With Relapsed or Refractory Small Cell Lung Cancer", the overall survival (time to death) was the primary efficacy endpoint. The death event would not be reported as AE/SAE and not recorded on AE/SAE case report form. It would be odd and inappropriate to record the AEs like "Death due to disease progression', 'death due to small cell lung cancer', 'death due to the underlying disease'. With Death being an efficacy endpoint, the death information should be recorded on separate case report form - Death Details form according to CDASH. 

Should death be reported as an event or an outcome of an AE/SAE?

Usually, when death event occurred, there were the adverse events leading to the death. The adverse event leading to the death will need to be reported as series adverse event with fatal outcome. The death is the outcome of a SAE and is not entered as an adverse event on its own. 

There are situations that the death is instantaneous or within very short period (for example one hour) of onset of symptoms or an unobserved cessation of life that cannot be attributed to a specific AE term. The death in this situation will be reported as Sudden Death or Sudden Death NOS on the AE case report form. 

If there are multiple ongoing AEs at the time of trial participant's death, should all these ongoing AEs be considered as SAEs with fatal outcome? 

This issue was discussed in a previous post "Recording the outcome for AE/SAE when multiple events contribute to Death". It is preferred in my opinion that a single AE/SAE should be identified as the primary cause for the death. This AE/SAE will have the fatal outcome recorded on the AE case report form. Other ongoing AEs at the time of death will have the AE outcome recorded as 'not recovered/not resolved"

How to fill out other fields on AE forms when death event occurs? 

As discussed above, for AE outcome, 'Fatal' should be selected for the AE/SAE that directly or primarily contribute to the study participant's death. 

For other ongoing AEs at the time of death, the AE outcome 'Not recovered or Not Resolved' should be selected.  

'Action Taken with Study Treatment' is another field on AE form and it is difficult to decide which choice should be selected when a death event occurs. 

At the time of death, there are two appropriate choices for 'Action Taken with Study Treatment": Dose Not Changed or Drug Withdrawn. Which choice to select depends on the sequence of the events: the last dose date/time in relevance to the death date/time. Assuming a study treatment with QD dose frequency, if the last dose date is the same as the death date or if the last dose date is one day prior to the death date, it is reasonable to assume that there is no dose withdrawn. Therefore, it is appropriate to select 'Dose Not Changed' for 'Action Taken with Study Treatment' field. On the other hand, if the last dose date is two days or earlier than the death date, it is appropriate to select 'Dose Withdrawn' for 'Action Taken with Study Treatment' field. 

Sentinel Dosing (Sentinel Subject) and Staggering Enrollment in First-in-Human (FIH) Clinical Trials

First-in-human (FIH) study is a type of clinical trial in which a new drug, procedure, or treatment is tested in humans for the first time. FIH studies take place after the new treatment has been tested in laboratory and animal studies and are usually conducted as phase I clinical trials. 

FIH study can be conducted in healthy volunteers (usually the case) or in patients.(in some special situations). Even though the new drug, procedure, or treatment has been thoroughly tested in pre-clinical studies before initiating the FIH study, the conservative approaches may still needed to be taken to ensure the safety of the study participants when designing the FIH study . 

FIH study can also be designed as phase 0 study or exploratory IND study as discussed in a previous post. 

FIH study may be designed as a single ascending dose (SAD) study where the healthy volunteers are enrolled and dosed in cohorts in dose-escalation fashion, i.e., the next dose cohort will only be enrolled after the safety data from the previous cohorts has been reviewed. FIH study may also be designed as dose-escalation study to identify the maximum tolerable dose (MTD) - such as the "3+3 design". 

Even with the SAD or dose escalation study designs, if it is uncertain there are still potential risks to the participants, additional precautions may be taken: sentinel dosing (sentinel subject) and staggering enrollment. 

Sentinel Dosing (Sentinel Subject): 

For the FIH study in healthy volunteers, the subjects are recruited to the clinical research unit (CRU, also called Phase I clinic). A cohort of subjects will be confined in the CRU to be dosed, observed, and evaluated. All subjects in the same dose cohort will be dosed at the same time. The study starts with the lowest dose cohort and then moves to higher dose cohorts. 

While dosing by cohort approach is usually safe, unexpected incidences can still occur. If the unexpected adverse events cause the harm to the study participants, it affects all participants in the entire cohort. Below are two examples where the phase I trial participants died or severely injured after receiving the experiment treatment in FIH Phase I studies. 

• Cytokine Storm in a Phase 1 Trial of the Anti-CD28 Monoclonal Antibody TGN1412
• French study’s death recalls 2006 UK clinical trial disaster - Were mistakes repeated?

To prevent this from happening, a strategy called sentinel dosing is often practiced so that one person in the first cohort of participants is dosed in advance of the full study or in advance of any full cohort. The very first subject who receive the sentinel dose is called 'sentinel subject'. 

Sentinel dosing was mentioned in EMA guidance "Guideline on strategies to identify and mitigate risks for first-in-human and early clinical trials with investigational medicinal products":

It is considered appropriate to design the administration of the first dose in any cohort so that a single subject receives a single dose of the active IMP (often known as sentinel dosing). Flexibility in this approach is allowed but should be on a risk-proportionate basis with a clear scientific rationale for any proposals not to use this strategy.

When the study design includes the use of placebo it would be appropriate to allow for one subject on active and one on placebo to be dosed simultaneously prior to dosing the remaining subjects in the cohort. This approach is expected for all single and multiple dosing cohorts, in order to reduce the risks associated with exposing all subjects in a cohort simultaneously. This sentinel approach may continue or also start to be appropriate at later stages of study design, e.g. on the steep part of the dose response curve, when approaching target saturation levels or the maximum clinical exposure levels defined in the protocol (see sections 7.5 and 8.2.9), in case of non-linear PK, or in light of emerging clinical signs or adverse events that do not meet stopping criteria. There should be an adequate period of time between the administration of treatment to these first subjects in a cohort and the remaining subjects in the cohort to observe for any reactions and adverse events. The duration of the interval of observation will depend on the PK and PD characteristics and the level of uncertainty associated with the product (see section 4). At the end of the observation period, there should be a clearly defined review of all available data for the sentinel subjects before dosing of further subjects in the cohort, with dose stopping rules in place to prevent further dosing if any rule is met (see also section 8.2.10).

Staggering Enrollment

Majority of phase I studies are conducted in healthy volunteers where the same cohort of subjects are recruited and confined at the clinical research unit (a single center) for dosing and post-dose measures and observations. In some situations (such as oncology studies, gene therapy trials, studies using human-plasma derived products), phase I studies are conducted in patients and are not ethical to be conducted in healthy volunteers. The patients will usually be recruited from multiple sites - so called multi-center phase I clinical trials. 

In multi-center phase I clinical trials, before multiple sites can start to recruit patients, a 'staggering enrollment' approach may be employed to minimize the potential harms caused by the innovative therapies. With the 'staggering enrollment' approach, after the first patient is enrolled, the second patient will only be enrolled after the first patient has been followed-up for a period of time and thoroughly evaluated for the safety measures. The third patient or the parallel enrollment will only be started after the second patient has been followed-up and thoroughly evaluated. 

The 'staggering enrollment' approach was used in the first-in-human trials in CAR-T trial and in gene therapy trials.  

The first CAR-T approval was for Novartis’s Kymriah (tisagenlecleucel) for the treatment of Acute lymphocytic leukemia (ALL). The tisagenlecleucel was originally developed by UPENN and FIH study was conducted by the UPENN. In their FIH study for CAR-T, the enrollment was staggering: 

“Staggered enrollment on the CNS3 cohort: infusion of any subsequent patient on the CNS3 cohort will be delayed until 21 days after the prior CNS3 patient’s infusion to allow for toxicity monitoring.”

For Bluebird’s Beti-cel in treatment of β-thalassemia patients requiring regular red blood cell (RBC) transfusions, their FIH trial also employed a staggering enrollment strategy:

“Initially, subjects with β-thalassemia major of the βE/β0 genotype will be enrolled in this study, and treatment will be staggered. The second subject will begin myeloablative conditioning only after the first subject 1) engrafts (defined as an absolute neutrophil count [ANC] ≥0.5 × 109/L for 3 consecutive days); and 2) has no LentiGlobin® BB305 Drug Product treatment-related serious adverse event (SAE) unexpected to occur with autologous HSCT. After Subject 2 meets these same criteria, parallel enrollment will be opened to additional subjects with the βE/β0 genotype.” 

'Staggering enrollment' may also be employed for the logistic reason. For a specific investigational site, the investigator and the study coordinator may not have the resource to enroll multiple patients all at once. They just don't have the manpower to do that. 'Staggering enrollment' approach allows the site to enroll one patient at a time. 

Drug Development for Rare Diseases - Public Workshops

Drug development for rare diseases is challenging, complex, but absolutely necessary. FDA has special programs to manage and encourage the drug development for rare diseases. 

According to "Rare Diseases at FDA" website, all three divisions (CDER, CBER, and CDRH) have special programs to support the drug development in rare diseases areas:

• CDER's Rare Disease Program
• CBER's Rare Disease Program

At "Regulatory Education for Industry (REdl)" meeting this week, there was a session "Partnering Across FDA to Advance Therapies for Rare Diseases" featuring three prosentations by FDA officers:

• Dr Retzky presented "FDA Rare Disease Updates: Incentives and Resources"
• Dr Lee presented "CDER's Perspective on Working Together with Our Rare Disease Partners within CDER and across FDA"
• Ms Vaillancourt presented "CBER's Perspective on Working Together with Our Rare Disease Partners across FDA"
• Dr Sherafat presented "Design Considerations for Clinical Trials in Rare Diseases"

FDA/CDER has an ARC program (Accelerating Rare disease Cures) aiming to drive scientific and regulatory innovation and engagement to accelerate the availability of treatments for patients with rare diseases. In year 1 of the ARC program, two important public workshops had been organized (webcasts can be watched following the links below):

• FDA/NIH Regulatory Fitness in Rare Disease Clinical Trials conference, May 16-17, 2022

CDER’s Rare Diseases Team and National Center for Advancing Translational Sciences
Focus on academic investigators and those looking to learn how to bridge the gap between academic investigation and the regulatory aspects of drug development

• FDA and Duke Margolis Virtual Public Workshop: Translational Science in Drug Development: Surrogate Endpoints, Biomarkers, and More, May 24-25, 2022

Focus on translational science and the development of surrogate endpoints

Clinical trials with external control, historical control, concurrent control, contemporaneous control, and synthetic control

Three critical features for modern clinical trials are control, randomization, blinding. For the golden standard of RCTs (randomized controlled clinical trials), a concurrent control group is critical. With recent advances in clinical trial designs, non RCTs such as real-world data (RWD)/real-world evidence (RWE), single arm trial, registry studies have been much discussed. The control group is now expanded to include concurrent control, external control, historical control, contemporaneous control. 

Concurrent Control: ICH E10 "Choice of Control Group in Clinical Trials" defined the concurrent control as the following: 

A concurrent control group is one chosen from the same population as the test group and treated in a defined way as part of the same trial that studies the test treatment, and over the same period of time. The test and control groups should be similar with regard to all baseline and on-treatment variables that could influence outcome, except for the study treatment. Failure to achieve this similarity can introduce a bias into the study. Bias here (and as used in ICH E9) means the systematic tendency of any aspects of the design, conduct, analysis, and interpretation of the results of clinical trials to make the estimate of a treatment effect deviate from its true value. Randomization and blinding are the two techniques usually used to minimize the chance of such bias and to ensure that the test treatment and control groups are similar at the start of the study and are treated similarly in the course of the study (see ICH E9). Whether a trial design includes these features is a critical determinant of its quality and persuasiveness.

Concurrent control is the feature of the RCTs and involves the randomization. The subjects are randomized into the test group or control group over the same period of time. 

External Control and Historical Control: ICH E10 "Choice of Control Group in Clinical Trials" defined the external control (including historical control) as the following:

External Control (Including Historical Control)

An externally controlled trial compares a group of subjects receiving the test treatment with a group of patients external to the study, rather than to an internal control group consisting of patients from the same population assigned to a different treatment. The external control can be a group of patients treated at an earlier time (historical control) or a group treated during the same time period but in another setting. The external control may be defined (a specific group of patients) or non defined (a comparator group based on general medical knowledge of outcome). Use of this latter comparator is particularly treacherous (such trials are usually considered uncontrolled) because general impressions are so often inaccurate. So-called baseline controlled studies, in which subjects' status on therapy is compared with status before therapy (e.g., blood pressure, tumor size), have no internal control and are thus uncontrolled or externally controlled (see section 2.5).

 Historical control is also external control. External control may or may not be historical control 

Contemporaneous Control may also be called contemporaneous cohort. In clinical trials with contemporaneous control group, subjects are recruited (not randomized) into the test group and the control group over the same period of time. The key idea is to compare subjects in the same time frame. For example, in a comparison of surgery versus chemotherapy for breast cancer, you wouldn't want to use surgery patients from 20 years ago as a control group to compare against a current chemo group. 

An great example of a clinical trial with a contemporaneous control group is a study assess the EVLP (ex-vivo lung perfusion) lung versus traditional (normal) lung in lung transplantations. In a non-randomized study "Extending Preservation and Assessment Time of Donor Lungs Using the Toronto EVLP System™ at a Dedicated EVLP Facility", according to the the study protocol, a contemporaneous control group was included to provide context for EVLP results and to inform control measures for future research. For every EVLP lung transplantation, a contemporaneous control lung transplantation with matched study center, single and double lung transplantation, lung allocation score. It is possible that for some EVLP lung transplants, the contemporaneous controls may not be identified which results in the large sample size in EVLP group than the contemporaneous control group. 

Once the donor lung is accepted following EVLP, the eligible recipient, who has provided written informed consent, and receives the lung transplant, is enrolled into the study. Patients who consent for the current EVLP , but receive a conventional (i.e., non-EVLP) lung transplant will be considered for a contemporaneous control group matched to the EVLP treatment group (66 subjects each). This matching will take place on a patient-by-patient basis and only after an EVLP subject has been enrolled at that Study Center. Investigators and their team will be notified by the Sponsor on a real-time basis of the specific matching criteria required for a control subject as EVLP subjects are enrolled. In order to be considered for eligibility, the control patient must “match” a priori to at least one EVLP subject who has already been enrolled at that Study Center based on the following criteria: SLT versus DLT and Lung Allocation Score Disease Diagnosis Group (LASDDG).

Contemporaneous control group is external, but concurrent control. Contemporaneous control group is similar to the matched control group in epidemiological case-control and cohort studies - similar statistical analysis approaches (such as conditional logistic regression) may be used for analyses.

Synthetic Control: synthetic control was discussed in a previous post "Synthetic Control Arm (SCA), External Control, Historical Control". Synthetic control includes subjects who are selected from historical clinical trials and who are on standard of case, and whose baseline characteristics match the current-day experiment group. Synthetic control is historical control, not concurrent control, but with matched baseline characteristics with the concurrent experiment treatment group. 

One Extra Point: 

One interesting discussion is about the control group in platform trial where multiple treatment arms are compared to the common control group. Since the different treatment arms may be added to or removed from the platform at different times, for a specific treatment - control group comparison, the control group may be not recruited over the same period of time. This issue was discussed in a NEJM paper "
Platform Trials — Beware the Noncomparable Control Group" and a JAMA paper "How to Use and Interpret the Results of a Platform Trial".

In platform trial, control group from a randomized trial may not be concurrent control.  

The Use of External Controls in FDA Regulatory Decision Making and Bayesian Borrowing

An good article by Jahanshahi et al "The Use of External Controls in FDA Regulatory Decision Making". The authors reviewed and summarized FDA regulatory approval decisions between 2000 and 2019 for drug and biologic products and summarized the pivotal studies that leveraged external controls, with a focus on select therapeutic areas.

The paper is open-access and available at "The Use of External Controls in FDA Regulatory Decision Making". 

The leading author is from BioMarin Pharmaceutical whose biological product Brineura for treatment of the rare Batten disease was approved by FDA based on an single arm study leveraging the external control from a historical registry. 

In the latest issue of the New England Journal of Medicine, Richeldi et al published a paper "Trial of a Preferential Phosphodiesterase 4B Inhibitor for Idiopathic Pulmonary Fibrosis". The trial was designed as a smaller (in sample size) trial with 2:1 randomization ratio by leveraging the data from the Placebo control groups in historical clinical trials. Bayesian borrowing (or Bayesian dynamic borrowing) approach was used for the analyses, as depicted below. While this is an example of successful use of external control in phase 2 study, it is unlikely that the same approach can be employed in their phase 3 pivotal studies mainly because that idiopathic pulmonary hypertension is a rare disease, but not rare enough. 

Power of the statistical graphs - an example of different ways for a Bar Chart

For the same set of data from the clinical trial, it is important to choose the appropriate statistical analysis methods. It is also important to choose the appropriate plot to display and visualize the data. The example below demonstrate the power of the statistical graphs (even the simple bar charts). 

A clinical trial is designed as a randomized, double-blinded, parallel three-arm study to evaluate the effect of high-dose and low-dose of an investigational drug in comparison with the Placebo. The outcome measure is NYHA Functional Class with four grades (I, II, III, and IV). At each post-baseline visit, function class is improved if it is shifted at least one grade toward the lower end, and functional class deteriorates if it is shifted at least one grade toward the high end. Suppose that at the end of the study, the outcome of the last visit can be organized as the following: 

	High-Dose (n = 200)	Low-Dose (n = 198)	Placebo (n = 202)
Improved	49 (24.5%)	37 (18.7%)	26 (12.9%)
No Change	139 (69.5%)	142 (71.7%)	148 (73.3%)
Deteriorated	12 (6.0%)	19 (9.6%)	28 (13.9%)

Both the dose (high-dose, low-dose, and placebo) and outcome (improved, no change, deteriorated) are considered to be the ordinal data. The statistical test with CMH gives a p-value of 0.0010 indicating the association between the dose groups and the outcome measures. 

The interesting thing is the data visualization - there are different ways to plot the data in the table above. What is the best way to plot the results in the table above?

If we are only interested in the proportion of subjects who have 'improved' functional class:

If we are only interested in the proportion of subjects who have 'deteriorated' functional class:

If we are interested in both the proportion of subjects who have 'improved' functional class and the proportion of subjects who have 'deteriorated' functional class: 

The bar chart can be arranged by outcome (improved and deteriorated) and the dose group (high-dose, low-dose, and placebo)

or the bar chart can be arranged by the dose group (high-dose, low-dose, and placebo) and then the outcome (improved and deteriorated). 

A stacked bar chart can be used to include all three outcome categories (improved, no change, and deteriorated):

The best approach is to place the outcome of 'improved' and 'deteriorated' functional class on the same vertical bar, but one above the 0-line and one below the 0-line. This approach assigns the 'improved' category as a positive value and the 'deteriorated' category as a negative value. 

These bar charts can be created using existing software such as SAS/Graphs, Microsoft Excel, and Graphpad Prism. It seems to be easier to use SAS to manipulate the data or obtain the aggregate data and then use Excel or Prism to create the charts. Prism is better and easier to use for creating charts for publications. 

Estimands, Estimator, Estimate, and Estimation

With the adoption of ICH E9(R1) "ADDENDUM ON ESTIMANDS AND SENSITIVITY ANALYSIS IN CLINICAL TRIALS TO THE GUIDELINE ON STATISTICAL PRINCIPLES FOR CLINICAL TRIALS",  the word 'Estimand' or 'Estimands' is appearing more and more in the clinical trial protocols, statistical analysis plans, regulatory documents, and publications. Just like the 'intention-to-treat' or 'ITT', the term 'Estimand' starts initially as a bizarre and confusing word, but will eventually be understood by clinical researchers who are working on clinical trials. Along with the word 'estimand', the words 'estimate', 'estimator', and 'estimation' should all be distinguished. 

According to ICH E9(R1) "ADDENDUM ON ESTIMANDS AND SENSITIVITY ANALYSIS IN CLINICAL TRIALS TO THE GUIDELINE ON STATISTICAL PRINCIPLES FOR CLINICAL TRIALS", they are defined as the following: 

Estimand: A precise description of the treatment effect reflecting the clinical question posed by the trial objective. It summarises at a population-level what the outcomes would be in the same patients under different treatment conditions being compared. 

Estimator: A method of analysis to compute an estimate of the estimand using clinical trial data.

Estimate: A numerical value computed by an estimator. 

 

Estimation is the process of finding an estimate or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable. 

For a clinical trial, 

• estimand (trial objective) is the target of estimation (for example, the treatment difference in change from baseline to Week 26 in FEV1) 
• estimator is the method of estimation (such as ANCOVA, Logistic regression, Cox regression) 
• estimate is the numerical result (such as LS mean difference, odds ratio, hazard ratio and their 95% confidence interval). 

Estimate is a rough calculation or guess while estimand is (statistics) that which is being estimated. p-value is from the statistical test is not an estimate (debatable) and naked p-values without the associated estimates should be explained with caution. 

The primary estimand (a precise description of the treatment effect reflecting the objective of the clinical trial) assessed effects regardless of treatment discontinuation or rescue interventions.

Using a cake as an example, estimand is the cake we want to make (what you seek), estimator is the recipe for making the cake (how you will get there), and estimate is the final result (what you get) - the cake we actually make - the estimate (final cake) should be close to the estimated (the cake we want to make). We turn our estimand into our estimate by applying an estimator. 

In a paper by Little and Lewis (JAMA, 2021) Estimands, Estimators, and Estimates, the estimated and the estimator were compared:

• The estimand compares outcomes that capture the main benefits and risks of treatments
• Estimators should summarize the causal effects of treatments in the sample of individuals in the study
• Estimands should summarize the causal effects of treatments in the target population
• The estimator should provide a valid and unbiased estimate of the study estimand

The paper was in response to the PIONEER 3 study: "Effect of Additional Oral Semaglutide vs Sitagliptin on Glycated Hemoglobin in Adults With Type 2 Diabetes Uncontrolled With Metformin Alone or With Sulfonylurea - The PIONEER 3 Randomized Clinical Trial". In PIONEER 3 trial, the primary estimand is the treatment policy estimand evaluating the treatment effect in change in glycated hemoglobin (HbA1c) from baseline to week 26 for all randomized patients regardless of trial product discontinuation or use of rescue medication. The estimator is to use analysis of covariance with a pattern mixture model using multiple imputation to handle missing data assuming the missing data mechanism is missing at random (MAR). The estimate is the actual value of the treatment differences as expressed as the estimated treatment differences and their 95% confidence intervals: "The 7- and 14-mg/d semaglutide dosages were superior to sitagliptin in reducing HbA1c from baseline at week 26 (estimated treatment differences of –0.3% [95% CI, –0.4% to –0.1%; P < .001] and –0.5% [95% CI, –0.6% to –0.4%; P < .001], respectively)"

In a Sanofi's trial "A randomized, double-blind, placebo-controlled study to evaluate the efficacy and safety of dupilumab in patients with severe steroid-dependent asthma", the primary estimand and the estimator (how the primary efficacy endpoint is analyzed) were discussed: 

The primary estimand is the intent-to-treat estimand, the treatment difference between dupilumab and control in the mean percentage reduction of OCS dose at Week 24 while maintaining asthma control of all patients in the ITT population no matter whether the patients discontinue treatment before Week 24 or not. To estimate the estimand, data of patients who permanently discontinue treatment will be incorporated in the primary analysis, and missing data due to patients dropping out from study will be handled by approaches specified in the missing data handling section below.

The primary efficacy endpoint will be analyzed using an analysis of covariance (ANCOVA) model. The model will include the percentage reduction of OCS dose at Week 24 as the response variable, and the treatment groups, optimized OCS dose at baseline, regions (pooled countries), and baseline eosinophil level subgroups (less than 0.15 Giga/L, greater than or equal to 0.15 Giga/L) as covariates. The treatment difference will be tested at the 2-sided significance level of alpha=0.05. Descriptive statistics for the primary efficacy endpoint will be provided, including the number of patients, means, standard errors, and least squares (LS) means by the treatment groups, as well as the difference in LS means and the corresponding 95% confidence interval (CI). The missing data (missing measures at Week 24) will be imputed using pattern mixture model by multiple imputation. 

In a paper by Harvard researchers,  "An Applied Researcher’s Guide to Estimating Effects From Multisite Individually Randomized Trials: Estimands, Estimators, and Estimates", the following conclusions are given:

"Defining an estimand is critical to the design, analysis, and interpretation of a multisite RCT. Even when one is interested in estimating an average treatment effect, careful consideration must be given to defining the target of inference. For example, the choice of estimand influences what formula is appropriate when conducting power calculations - yet many write-ups of power calculation are ambiguous or silent with regards to the estimand chosen. relatedly, registering studies and creating analysis plans are becoming the norm when conducting RCTs, yet even newly created registries do not require an estimand be defined. Consequently, readers of an analysis plan are left to assume an implied estimand based on the power calculation formula used and/or estimator selected. Similarly, many scholarly articles and reports do not state a target of inference, making it difficult to assess whether the chosen estimator is appropriate and obscuring the goal of the research. We recommend defining the estimand(s) early in the research process and clearly stating them in important products."

Randomization and elements of randomization specifications

Randomization is the process of assigning trial subjects to treatment or control groups using an element of chance to determine the assignments to reduce bias. Randomization is the most critical feature of the RCT (randomized, controlled trials). In FDA's Good Review Practice: Clinical Review of Investigational New Drug Applications, the randomization is defined as the following: 

In the context of clinical trial design, randomization is defined as the allocation of patients to the investigational drug and control arms by chance. Randomization is intended to prevent any systematic difference between patients assigned to the treatments being compared and is a critical assumption for valid statistical comparisons. It is also intended to produce groups that are comparable (statistically balanced) with respect to both known and unknown factors. 

Randomization Schedule (also called a randomization scheme) is a list of randomization numbers and the corresponding treatment assignments in a data set (or in a printout in the early days). Randomization Schedule can be generated using SAS Proc Plan. A SUGI paper"Generating Randomization Schedules Using SAS Programming" I wrote 20 years ago is still applicable. 

Three steps for generating the randomization schedule for use in clinical trials: 

• Create randomization specifications according to the study protocol requirements
• Create and validate dummy randomization schedule for review and approval
• Create and validate the final randomization schedule - the final randomization schedule for implementation

The dummy randomization schedule and the final randomization schedule have the same display but are generated with different random seeds (therefore different treatment assignments). The dummy randomization schedule can be reviewed by the study team and the final randomization schedule can only be distributed to the designated recipients who are unblinded to the treatment assignments. 

Here is an example randomization specification: 

Here are the elements for the randomization specifications: 

Study Design: clinical trial design dictates how the subjects are assigned to receive different study treatments. In clinical trials with parallel design, subjects are randomized to receive the treatments; in clinical trials with cross-over design, subjects are randomized to different treatment sequences. 

The study design will also include a randomization strategy: 

Fixed randomization: 

• fixed-randomization scheme (rarely used)
• block randomization
• stratified randomization, 

Dynamic randomization: 

       Adaptive randomization. 

See FDA's Good Review Practice: Clinical Review of Investigational New Drug Applications for definitions of these different types of randomizations. 

Blind and blindness: concealing treatment assignments and treatment allocations. 

Block: Block randomization works by randomizing subjects within blocks such that within each block, the # of subjects is balanced between treatment groups or according to the randomization ratio. 

Block Size: The size of each block. Block sizes must be multiples of the number of treatments and take the allocation ratio into account. For 1:1 randomization of 2 groups, blocks can be sizes 2, 4, 6 etc. For 1:1:1 randomization of 3 groups or 2:1 randomization of 2 groups, blocks can be sizes 3, 6, 9 etc. 

If the randomization is by site, to prevent the potential unblinding/guessing, the block size can be set up as variable for different blocks or is not revealed to the investigators and study team. With central randomization, potential unblinding is less of a concern and the block size can be the smallest multiples (for 1:1 randomization of 2 groups, the block size can be 2). 

Number of Blocks

Total Number of Randomizations: Total number of randomization numbers to be generated. Total number of randomizations = Number of blocks x Block size. Usually, randomization numbers more than the protocol-specified sample size are generated to make sure that there is a sufficient number of randomizations in the situation that the sample size may be increased or randomization errors that results in some randomization numbers not being used. If the study protocol specifies 300 subjects to be randomized, it may be good to generate 600 randomizations. 

Strata and Stratification Factors: Stratification factors are those known factors that may have an impact on treatment responses. Stratification factors are the known confounders. The most common stratification factor is the baseline disease severity which usually has an impact on the treatment responses. When stratification factors are specified, stratified randomization is employed to prevent imbalance between treatment groups for known factors that influence prognosis or treatment responsiveness. The randomization schedule is essentially generated for each stratum. 

See previous posts "Restricted randomization, stratified randomization, and forced randomization"; "Minimization Algorithm to Achieve Treatment Balance across Strata in Stratified Randomization", and "Handling Randomization Errors in Clinical Trials with Stratified Randomization"

Randomization Ratio (or allocation ratio): The ratio for treatment groups. The typical randomization ratio is balanced: 1:1 ratio for two treatment groups (if the block size is 2, for every 2 subjects randomized, there will be one assigned to group A and one assigned to group B); 1;1:1 ratio for three treatment groups, ... The randomization ratio can also be unbalanced such as 2:1 (if the block size is 3 (minimal), for every three subjects randomized, there will be two assigned to group A and one assigned to group B) and 3:1,...  FDA's Good Review Practice: Clinical Review of Investigational New Drug Applications described the randomization ratio (allocation ratio) as the following: 

Allocation of patients to treatment and control arms can be uniform or nonuniform. Uniform allocation (i.e., equal numbers allocated to each arm) is the usual practice and provides the most statistical power for a given total sample size. Nonuniform allocation  may lower costs (if one arm is substantially more expensive) and improve recruitment (if one arm is generally preferred) and may increase the size of the exposed patient safety database. In general, the loss of statistical power in seeking to detect a difference between treatments going from uniform allocation to 2:1, or even 3:1, is fairly small; however, as more imbalanced allocation occurs, power drops off more rapidly. A special case is where a trial seeks both to show effectiveness versus placebo and to compare the test drug with an active control. In that case, it usually is necessary for the active treatment groups to be substantially larger to examine the smaller differences between the active treatments. 

Randomization Number: a series of sequential numbers corresponding to treatment assignments. 'randomization number' is not random, the associated treatment assignments are random. 

Randomization can be recorded in the database and serve as the subject identifier (same as the subject number). Seeing the randomization number will not unblind the subject's treatment assignment.  

Treatment Code: short description or abbreviation for long treatment descriptions. Treatment code can be just the letters (such as A = Active; P = Placebo). 

Treatment Description: the detailed description of the treatment groups. It can be just 'Active', 'Placebo' or more descriptive as "Inhaled drug X BID', 'Inhaled Placebo BID'.

Dummy Randomization Schedule: also called surrogate randomization schedule - the randomization schedule for review and approval purposes. The dummy randomization schedule should have exactly the same features as the final randomization schedule except that a different random seed is used (therefore, the treatment assignments are different). 

Random Seed: A number (integer) used to initiate a pseudorandom number generator. Random Seed is a number used in SAS Proc Plan to generate the randomization schedule. Random Seed needs to be specified in the program in order to reproduce the same randomization schedule. 

After dummy randomization schedule is reviewed and approved, a final randomization schedule can be generated for implementation by changing the random seed. 

Randomization Envelopes: Envelopes that contain the treatment assignment information. The outside of the envelope contains the randomization number, and the inside of the envelope contains the randomization number and the corresponding treatment assignments and treatment descriptions. Randomization envelopes were used in the randomization process in the early days. The randomization process using randomization envelopes is now replaced with the Interactive Response Technology (IRT) including the Interactive Web Response System (IWRS) or Interactive Voice Response System (IVRS). 

Central Randomization is the opposite of randomization by the site. When a subject is eligible to be randomized, the site will contact a centralized contact (usually the computer system, IRT) to obtain the next available randomization number in the corresponding stratum regardless of the individual sites.

References: SAS/STAT User's Guide The PLAN Procedure

Common Issues in Implementing Randomization and Blinding

Randomization and blinding are two techniques to help prevent (conscious or unconscious) bias in clinical trials and they are the cornerstone of the randomized, controlled clinical trials (RCTs) and in FDA's terms, the cornerstone of the adequate & well-controlled clinical trials (A&WCs). As stated in FDA's Good Review Practice: Clinical Review of Investigational New Drug Applications:

Randomization and blinding are the two principal means of reducing bias and ensuring validity of trial conclusions. Randomization helps protect against the possibility that differences between groups at baseline will lead to outcome differences that might mistakenly be attributed to drug effect. Blinding protects against the possibility that differences in the on-trial treatment or assessment of subjects will lead to spurious outcome differences that are mistakenly attributed to a drug effect. 

In the context of clinical trial design, randomization is defined as the allocation of patients to the investigational drug and control arms by chance. Randomization is intended to prevent any systematic difference between patients assigned to the treatments being compared and is a critical assumption for valid statistical comparisons. It is also intended to produce groups that are comparable (statistically balanced) with respect to both known and unknown factors. 

While every effort is made to prevent the mistakes in implementing the randomization, it is inevitable to have the randomization errors and mistakes here and there. Here are some of the randomization errors that may be seen in clinical trials. 

Ineligible subjects are randomized: Clinical trials contain a screening period for verifying the eligibility of the study participants. All inclusion and exclusion criteria are checked during the screening period. If a subject meets all inclusion and exclusion criteria, the subject is eligible to be randomized. Sometimes, subjects are thought to be eligible for randomization, but only, later on, are found to be ineligible for one or more entry criteria. When ineligible subjects are randomized into the study and receive the assigned study treatments, the subjects are considered to be in the study. According to the intention-to-treat principle, the subjects will be included in the analyses regardless of the violation of the inclusion or exclusion criteria. If the critical criteria that are violated have an impact on the efficacy evaluation, the subjects may be excluded from the per-protocol population and sensitivity analyses are conducted with the per-protocol population to assess the robustness of the results from the primary analysis. 

Choosing the wrong stratum for randomization: For clinical trials with stratified randomization, the randomization is executed within each stratum. When a new subject is eligible to be randomized, the next available randomization number in the corresponding stratum (for example, based on the subject's gender, baseline disease severity category,...) is allocated to the subject. 

It is not uncommon that the investigational sites to select an incorrect stratum for the randomization especially when the strata information requires additional derivation and calculation, for example, if a  subject with or without using one class of background medications is a stratification factor, the information about the use one class of background medication may need to be derived. 

If a randomization stratification factor is measured more than one time, which measure will be used for randomization needs to be clearly stated in the protocol. If a spirometry parameter (for example, % predicted FEV1 >= 50% versus <50%) is used as a randomization stratification factor and spirometry tests are performed at both screening and baseline visits, the protocol needs to be specific regarding whether the results from screening visit or the baseline visit will be used for randomization - typically, the measures at the baseline visit should be used for randomization. If a laboratory parameter is used for randomization and there are both local lab and central lab, the protocol needs to be specific regarding which lab results will be used for randomization - typically the central lab results at the baseline will be used for randomization unless the central lab results can not be obtained in time for the randomization. 

See previous post "Handling Randomization Errors in Clinical Trials with Stratified Randomization"

Randomize the patients too early before all eligibility criteria are met: The investigator rushed to go to the randomization system (IRT) and randomized the subject to trigger the downstream activities, then realized that one or more screening results were still pending. 

Once the subject is randomized, it can’t be undone in the randomization system (IRT system). However, the site can hold on to the randomization information obtained and wait for the last pieces of the screening results to confirm the eligibility. If the last piece of the screening results confirms that the subject is eligible to be randomized, the previously obtained randomization information will then be used. The subject can move on to initiate the assigned study treatment. If the last piece of the screening results indicates that the subject is ineligible to be randomized, we will then need to decide if the subject is allowed to be in the study. If so, it will become the situation mentioned in the previous section "Ineligible subjects are randomized". 

In either situation, a protocol deviation needs to be recorded to document this incident. 

The PI was practicing the randomization system to see how the randomization works but accidentally randomized the subject in a live system. In this situation, there was no actual and real subject to be randomized. The subject information entered into the randomization system was not real, but one of the randomization numbers was assigned and treatment assignment was used. 

While this subject may remain in the randomization system (IRT), the subject is fake and should be removed from the downstream clinical database. This is usually a rare event, therefore, has no big impact on the integrity of the original randomization. 

Randomization system (IRT system) is down at the time of randomization or the internet is down: sometimes, the randomization needs to be performed immediately after the last eligibility criterion is confirmed. It is critical to have immediate access to the randomization information in order to randomize the subject in time for initiating the randomized treatment. However, it could happen that the IRT system is down or the internet is down when the randomization number and treatment assignments are needed. 

If this is a situation, the advice is to have a backup manual randomization system (for example, calling an unblinded person or group). 

Dispense the incorrect drug kit: nowadays, the randomization system is embedded in the system for clinical trial supplies (IRT system). In addition to the treatment assignments, a separate drug kit list will be generated. When a subject is randomized and a treatment group is assigned, the drug kits that are corresponding to the assigned treatment will be allocated and dispensed to the subject. 

See a previous post "Monitoring the double-blind study: unblinded pharmacist, unblinded monitor, and drug kit"

Due to human error, it is possible to have the correct randomization information but dispense the incorrect kit numbers. When this happens, it is adverse to verify with the clinical trial supply manager (who are unblinded) if the incorrect drug kit is for the same treatment group as the drug kit that is supposed to be dispensed (Don't communicate about the actual treatment group). It is less an issue if the incorrect kit numbers are in the assigned treatment group.

If the incorrectly dispensed drug kits are not in the assigned treatment group, the subjects received the incorrect treatment. For the statistical analysis, the subject will be included in the intention-to-treatment analysis and will be included in the randomized treatment group  (so-called 'as randomized). The subject can be excluded from the per-protocol population for sensitivity analysis. 

Recording the randomization date/time (local time versus backend system time): When a subject is randomized in the IRT system, a randomization message or printout, or randomization report will indicate the subject number, randomization number, the stratification factors used for randomization, randomization date/time. The investigator can record the randomization information in the case report form. Only blinded information can be included in the randomization report. 

One issue for this report is the randomization date/time - is it based on the IRT system date/time or the local date/time?  The local date/time should be used as the randomization date/time. If the IRT system is located in the UK and the subject is randomized in the US, the local and system time can differ in 5-8 hours. Local date/time, not the system date/time should always be used as the randomization date/time.  

The same subject is randomized twice (unless it is the micro-randomized trial)

The majority of these randomization errors that occurred in the study were not included in the publications and regulatory submissions - the randomization issues appear to be less than what actually occurred. Some of the examples of the randomization errors can still be found in the literature: 

• This article by Downs et al "Some practical problems in implementing randomization" documented some situations with randomization errors. 
• in a Takeda study SAP, a section was included to describe the handling of randomization errors. 

• in the recent FDA's advisory committee meeting to discuss Amylyx pharmaceuticals' ALS drug, the randomization issues in their randomized trial (CENTRAUR trial) were discussed. Here is what in the presentations: 

"To start, at the beginning of CENTAUR, a randomization implementation problem was identified and addressed by the unblinded statistician. Let’s walk through the details. In CENTAUR, kits were shipped one by one after successful screening visits. While preparing for the first Data Safety Monitoring Board meeting in November 2017, the unblinded statistician found that the initial 18 study kits shipped were all active. This was due to an error at the distribution center. They proceeded to instruct the distribution center to balance these 18 kits by shipping a block of 9 placebo kits to maintain randomization. After correction, the 2:1 active:placebo ratio was maintained. The unblinded statistician notified Amylyx of this issue in January 2020, two months after study unblinding in November 2019. Participants, investigators, and study staff were never unblinded due to this error. Upon notification, Amylyx initiated a thorough investigation of the root cause, in consultation with the unblinded statistician and the distribution center. Amylyx also consulted with external statisticians to determine the best approach to assess the impact. The statisticians recommended a sensitivity analysis to exclude the participants affected by the error."

•  in a paper by Douglas et al, "Fluid Response Evaluation in Sepsis Hypotension and Shock: A Randomized Clinical Trial", 4 subjects with randomization errors were excluded from the intention-to-treat analyses even though there was no mention what kind of randomization errors were.
• in a paper by Martinez et al "Treatment of Persistent Cough in Subjectswith Idiopathic Pulmonary Fibrosis (IPF)with Gefapixant, a P2X3 Antagonist, in a Randomized,Placebo-Controlled Clinical Trial", a programming error in IWRS caused the randomization errors.