Randomized Withdrawal Design - Examples for Defining the Criteria for Run-in and Randomized Withdrawal Periods

In a previous post, we discussed the "Randomized Withdrawal Design and Randomized Discontinuation Trial". The randomized withdrawal design and the randomized discontinuation design may be used interchangeably. The randomized withdrawal design is one of the clinical trial designs with enrichment strategy and is more efficient design if it is applied in the appropriate situation.

The diagram for a typical randomized withdrawal design will be something like below (with efficacy measure in the run-in period).

or with safety or tolerability as the measure in the run-in period.

The randomized withdrawal design contains a run-in period and a randomized, controlled period. For each period, the criteria will need to be defined. For open-label run-in period, the criteria are needed to define what is considered as 'responder' (for efficacy measure) or 'tolerable' (for safety measure). For the randomized, controlled period where the formal hypothesis testing is based on, the study endpoint will need to be defined.

The table below listed some examples of clinical trials using the randomized withdrawal design. The criteria for the run-in period and for randomized withdrawal period are listed. 

	Criteria for Responder or tolerability	Endpoint for Randomized Withdrawal Period
ICE Study (immunoglobulin in CIDP)	10-point INCAT score improve by 1 point	Time to relapse where the relapse was defined as 10-point INCAT score worsen by 1 point
Droxidopa for Symptomatic Neurogenic Orthostatic Hypotension	based on symptom and BP response where the response is determined by improvement of at least one point on a symptom question (Orthostatic Hypotension Symptom Assessment (OHSA) Item 1) and an improvement in SBP of at least 10 mmHg at 3 minutes post-standing]	the mean change from Randomization (Visit 4) to the End of Study Visit (Visit 5) in the OHSA Item 1 (dizziness, lightheadedness, feeling faint or feeling like you might black out) score
Nifedipine for vasospastic angina	Stable	recurrence of severe vasospastic angina leading to study withdrawal
Gamma-hydroxybutyrate (GBH, sodium oxybate) for treatment of cataplexy	Median Attacks every 2 weeks	Median Attacks every 2 weeks
Double-blind, Placebo-controlled, Randomised Withdrawal, Extension, Safety and Efficacy Study of LDX in Children and Adolescents Aged 6-17	Completed a minimum of 4 weeks of double-blind treatment, reached Visit 4 and completed the 1-week post-treatment washout in the antecedent study (SPD489-325), without experiencing any clinically significant AEs that would preclude exposure to LDX	Treatment failure defined as 50% increase (worsening) in Attention Deficit Hyperactivity Disorder Rating Scale (ADHD-RS-IV) total score and greater than and equal to 2 point increase (worsening) in the Clinical Global Impression-Severity of Illness (CGI-S) score observed at any visit during the randomized withdrawal period compared to the respective scores at baseline of randomized withdrawal period.
A double-blind, placebo-controlled, randomized withdrawal study of lurasidone for the maintenance of efficacy in patients with schizophrenia	To achieve and maintain clinical stability for at least 12 weeks during the open-label stabilization phase and to have remained on a stable dose of lurasidone for four weeks prior to randomization. Clinical stability was defined as a PANSS total score less than or equal to 70, with PANSS item scores less than or equal to 4 on all positive subscale items and the item for "uncooperativeness”, (item G8), and a CGI-S score less than 4.	Time to relapse (based on Kaplan–Meier survival analysis), with relapse defined as greater than and equal to 1 of the following during the double-blind phase: (1) An increase of greater than and equal to 25% from double-blind baseline in PANSS total score and CGI-S worsening of greater than and equal to 1 point for two consecutive visits no more than ten days apart. (2) At any single visit, a PANSS item score of greater than and equal to 5 (moderately severe) on hostility or uncooperativeness, or a PANSS item score of greater than and equal to 5 on two or more items of unusual thought content, delusions, conceptual disorganization, or hallucinatory behavior. (3) Initiation of supplemental treatment with an antipsychotic agent other than lurasidone, an increased dose of an antidepressant or mood stabilizer, an increase in lorazepam (or benzodiazepine equivalent) dose by greater than and equal to 2 mg/d for at least 3 days, or electroconvulsive therapy. (4) Insufficient clinical response or exacerbation of underlying disease reported as an adverse event, as determined by the study investigator. (5) Deliberate self-injury or repeated aggressive behavior, active suicidal or homicidal ideation or attempt. (6) Psychiatric hospitalization due to worsening schizophrenia. Secondary
A Randomized Withdrawal, Placebo-Controlled Study Evaluating the Efficacy and Tolerability of Tapentadol Extended Release in Patients With Chronic Painful Diabetic Peripheral Neuropathy	Patients who tolerated tapentadol ER and had 1-point improvement in average pain intensity from the pre-titration evaluation period to the last 3 days of the open-label titration period were randomly assigned (1:1) to receive tapentadol ER or placebo during a subsequent 12-week double-blind maintenance phase	The primary efficacy endpoint was the mean change in average pain intensity from baseline to week 12
Long-term Maintenance of Response Across Multiple Fibromyalgia Symptom Domains in a Randomized Withdrawal Study of Pregabalin	meeting response criteria for pain [Z50% reduction in pain 100-mm Visual Analog Scale (VAS) score from OL baseline] and PGIC (self-rating of much improved or very much improved) at the end of the OL treatment phase	Time to LTR (loss of therapeutic response)
The randomized discontinuation trial: a phase II design to assess growth-inhibitory agents	Radiologic stable	Fraction with radiologic stable disease

FDA Guidance: Expansion Cohorts: Use in First-In-Human Clinical Trials to Expedite Development of Oncology Drugs and Biologics

Two years ago, I had a post discussing "Dose Cohort Expansion Study - A Never Ending Phase I Study Design". Last month, FDA officially issued the guidance for industry "Expansion Cohorts: Use in First-In-Human Clinical Trials to Expedite Development of Oncology Drugs and Biologics".

With this guidance, we expect that more sponsors will adopt this study design in oncology trials and beyond the oncology trials.

The dose cohort expansion study design completely changed the traditional phased approaches for clinical trials for drug development. The phases I, II, III, IV are no longer critical.

Splitting p-value and estimate of treatment difference

When we perform the statistical test to compare the difference between two treatment groups, we usually construct a test statistic, calculate the treatment difference, and then obtain the p-value corresponding to the test statistic and the treatment difference.

For example, for a study with primary efficacy endpoint of proportion of subjects with hemostasis, Treatment difference was estimated using risk ratio and the corresponding p-value was calculated to indicate if the risk ratio is statistically significant. 

Another example is for a study with 6-Min walk distance (6MWD) as the primary endpoint. The treatment difference was estimated using least-squares mean difference. The corresponding p-value was calculated using the analysis of covariance (ANCOVA) approach. 

In these examples, the p-value and the estimate of the treatment difference were from the same test statistic. 

We see many examples where two different methods are used for estimating the treatment difference and for calculating the p-value - I call it 'splitting the p-value and the estimate of the treatment difference'. Here are two situations where this splitting situation occurs.

Analysis of Time to Event: log-rank test for calculating the p-value and proportional hazard model for estimating the hazard ratio 

In clinical trials with time to event endpoint, it is very common to provide the Kaplan-Meier estimate and calculate the p-value using log-rank test - a non-parametric method. The treatment difference or magnitude of treatment effect is usually measured using hazard ratio. Kaplan-Meier estimate does not give an estimate of the hazard ratio. The hazard ratio needs to be estimated using the Proportional Hazard model (or Cox regression model). 

In a study by Sitbon et al, the primary efficacy endpoint was a composite endpoint of time to death or a complication related to PAH. The hazard ratio and p-value were provided in the primary efficacy table below. However, it needs to be noted that two different methods were used to calculate the hazard ratio and the p-value. As stated in the Statistical Analysis section, the statistical methods were provided as the following:  

"In time-to-event analyses, end points were estimated with the use of the Kaplan–Meier method and were analyzed with the use of the log-rank test. Hazard ratios with 99% confidence intervals (for primary and secondary end points) and 95% confidence intervals (for exploratory end points) were estimated with the use of proportional-hazard models."

 

p-value corresponding to the hazard ratio can also be obtained from the proportional hazard model, but is usually not presented in the place where p-value from the log-rank test is provided - to avoid the confusion about two different p-values. 

Why do we present the hazard ratio from one method and p-value from another method? why can't we present both the hazard ratio and the corresponding p-value from the proportional hazard model? 

   

Wilcoxon Rank Sum Test to calculate the p-value and Hodges-Lemman method to calculate the difference in median

Wilcoxon is also called Mann Whitney U Test and is a non-parametric method to compare the difference in medians for non-normal distributed data. Wilcoxon rank sum test converts the original data into ranks and the p-value is calculated to compare the total ranks between groups. However, the statistics of ranks has no meaning in measuring the magnitude of the treatment effect. Therefore, when the Wilcoxon method is used to calculate the p-value, a different method needs to be employed to estimate the treatment difference (magnitude of the treatment effect). Hodges-Lehmann method is now commonly used to estimate the treatment difference - location shift in medians between two treatment groups. 

Here is a link to FDA's statistical review for Xermelo (telotristat ethyl) oral tablets in indication of  Carcinoid Syndrome. p-value was calculated from Wilcoxon rank test and treatment difference (location shift in median) and confidence interval were calculated from Hodges Lehmann method.

The primary efficacy endpoints in studies LX301 and LX303 were analyzed by the blocked 2- sample Wilcoxon rank sum statistic stratified by the baseline urinary 5-HIAA levels (≤ upper limit of normal reference range [ULN], >ULN, and Unknown). Descriptive statistics of the primary endpoints and the Hodges-Lehmann estimator of location shift with its respective CLs were reported for each comparison.

In a paper by Jing et al, "Efficacy and Safety of Oral Treprostinil Monotherapy for the Treatment of Pulmonary Arterial Hypertension A Randomized, Controlled Trial", the endpoint of 6MWD was analyzed using Wilcoxon method for p-values and using the Hodges-Lemman estimator for the treatment effect (location shift in medians).

Geriatric Investigational Plan - Clinical Trials for Elderly

Every clinical trial should have a protocol and the protocol needs to include the inclusion/exclusio criteria to define the study population (i.e., who can participate in the trial). The first inclusion criterion is usually the age limit. In many clinical trials sponsored by AstraZeneca, the 'Ages Eligible for Study' was defined as "18 Years to 130 Years (Adult, Older Adult)". Here are links to entries in clinicaltrials.gov for two of these studies: NEPTUNE study and MYSTIC study The upper limit of 130 years old for the study triggered me to look into the age range for clinical trials in adult population (especiaily the upper limit in elderly patients).

The clinical trials in the pediatric population and the age groups in the pediatric population have been discussed in previous posts.

• Age Group in Pediatric, Perinatal or Preterm, and Geriatric Subjects
• Pediatric use and geriatric use of drug and biological product

The reason to have an upper limit of age for study participants is to have a homogeneous population for the study to maximize the chance to have a positive study.  Nowadays, many trials do not set an upper limit for age. An 80-85-year-old individual can be very healthy and an 80-85 years old patient can participate in clinical trials without problems.

For pediatric population, in order to obtain the indication or label, a separate set of clinical trials need to be conducted. There are many requirements and rules for pediatric clinical trials. Regulatory agencies encouraged the sponsors to do clinical trials in pediatric population – pediatric investigational plan (PIP).

Similarly, the safety and efficacy of the specific drug product should also be studied in the geriatric population. The safety and efficacy of a product may be different between the geriatric population and the general adult population. There was an ICH E7 STUDIES IN SUPPORT OFSPECIAL POPULATIONS:GERIATRICS and its Questions and Answers. FDA has a guidance on "Content and Format for Geriatric Labeling". Below are some additional discussions about the clinical trials in the geriatric population.

• Disparate Inclusion of Older Adults in Clinical Trials: Priorities and Opportunities for Policy and Practice Change
• Why Do Clinical Trials Need Older and Diverse Participants?
• https://www.fda.gov/downloads/RegulatoryInformation/LawsEnforcedbyFDA/SignificantAmendmentstotheFDCAct/FDASIA/UCM410474.pdf
• FDA-ASCO Public Workshop: Geriatric Oncology Workshop
• Clinical Discussion of Specific Clinical Discussion of Specific Populations
• GAO 2007 Prescription Drugs: FDA Guidance and Regulations Related to Data on Elderly Persons in Clinical Drug Trials

For a study in adult including elderly patients, the typical criteria for age will be 18 years and older with no upper limit for age. In this way, the geriatric subjects can be enrolled into the study if the subject meets all other inclusion/exclusion criteria and the very older patients can be enrolled into the study if they are healthy enough. However, if we look at the various studies, different sponsors use different criteria for age in their studies.

Using Phase III studies in NSCLC, here are some examples from clinicaltrials.gov. We can see that different clinical trials use different ranges for age. 

Study	Inclusion Criterion for Age
A Study of Nivolumab + Chemotherapy or Nivolumab + Ipilimumab Versus Chemotherapy in Patients With EGFR Mutation, T790M Negative NSCLC Who Have Failed 1L EGFR TKI Therapy (CheckMate722)	18 Years and older (Adult, Older Adult)
Evaluate the Efficacy and Safety of IBI305 in Patients With Advanced or Recurrent Non-squamous NSCLC	18 Years to 70 Years (Adult, Older Adult)
MIL60 Versus Bevacizumab in Patients With Treatment-naïve Non-squamous Non-small Cell Lung Cancer	18 Years to 75 Years (Adult, Older Adult)
A Study Comparing MB02 and Avastin® in Subjects With Stage IIIB/IV Non-Squamous Non-Small Cell Lung Cancer (NSCLC) (STELLA)	18 Years to 80 Years (Adult, Older Adult)
Effect of Tumor Treating Fields (TTFields) (150 kHz) Concurrent With Standard of Care Therapies for Treatment of Stage 4 Non-small Cell Lung Cancer (NSCLC) Following Platinum Failure (LUNAR)	22 Years and older (Adult, Older Adult)
Study of ONO-4538 in Non-Squamous Non-Small Cell Lung Cancer	20 Years and older (Adult, Older Adult)

Whether or not the upper limit is set to 70, 75, 80 years old or have no upper limit, if the drug is approved, the indication for the drug will be the same and the only difference may be the descriptions in section 8.5 of the drug label per FDA's guidance "Labeling for Human PrescriptionDrug and Biological Products –Implementing the PLR Content and Format Requirements".

Sample Size Calculation Without Considering Dropout Rate

When planning for clinical trials, sample size calculation is a two-step or three-step process. For clinical trials with continuous variable or categorical variable as primary efficacy endpoint, the sample size calculation is usually a two-step process: the step 1 is to calculate the sample size based on the effect size / standard deviation for continuous variable; difference in rate/proportions and the control group rate/proportion for categorical variable. The step 2 is to apply the dropout rate to calculate the number of subjects needed to be randomized. I had an earlier post “sample size considering the dropout rate” to state that the calculation to account for the dropout rate should be by dividing (1-dropout rate), not by multiplying (1+dropout rate).

For clinical trials with time to event variables, the sample size calculation is usually involved in three steps:

• Step 1: Estimate the number of events needed based on hazard ratio, median survival time, or event rate at a specified time frame
• Step 2: Estimate the number of subjects needed to obtain the required number of events based on the accrual time, the follow-up, or the overall study duration
• Step 3: Estimate the total sample size by considering the dropout rate.

Applying the dropout rate in clinical trials with time to event endpoint is not straightforward. The dropout rate is usually not a constant throughout the study (during the accrual period and during the follow-up period).

To calculate the sample size for studies with time to event variables, I usually use Cytel’s EAST software. To deal with the dropout rate, the EAST software user manual suggests a ‘trial and error’ method. With this method, we will need to provide an initial dropout rate (expressed in ‘probability of dropout’ or ‘hazard for dropout’). We then check the summary of the sample size estimation. The number of dropouts is displayed, and the dropout rate can easily be calculated by dividing the number of dropouts by the number of subjects. If the calculated dropout rate is different from the assumptions, we can then go back to revise the initial input – do this several times until the calculated dropout rate from EAST matches the assumed dropout rate.

I recently run into two scenarios where the sample size calculation does not need to consider the dropout rate.

• Sample size calculation for an animal study (pigs for example) with a time to event variable. The pigs used in the experiment will be confined in a facility. There is no lost-to-follow-up or anything situation like that. There is no need for us to consider the dropout rate if we try to estimate the number of pigs needed for an experiment. 

• Sample size calculation for a metastatic non-small cell lung cancer (NSCLC) study with overall survival as the primary efficacy endpoint. The median survival time for metastatic NSCLC patients is usually short. In other words, the mortality rate for NSCLC patients is high. In this situation, the number of subjects who lost to follow-up is generally small (less than 5%) and we may not need to apply the dropout rate in calculating the number of subjects to be randomized. A friend of mine who is an expert in NSCLC clinical trials told me that in their studies with OS as primary efficacy endpoint, they will calculate the number of death events needed, and then simply add 25-30% more patients on top of the number of death events to account for a small proportion of subjects who may live much longer. For example, based on the hazard ratio, accrual time, and follow-up time, if the calculated number of death events is 200 events, we can simply have a sample size of 250-260 subjects randomized (25-30% more than the number of death events). 

Here are some recent trials from New England Journal of Medicine where there was no mention of dropout rate in sample size calculations:

• Durvalumab after Chemoradiotherapy in Stage III Non–Small-Cell Lung Cancer and the study protocol
• Nivolumab versus Docetaxel in Advanced Squamous-Cell Non–Small-Cell Lung Cancer and the study protocol
• Atezolizumab for First-Line Treatment of Metastatic Nonsquamous NSCLC and the study protocol
• Dimopoulos 2018 Phase 3 Trial of Ibrutinib plus Rituximab in Waldenström’s Macroglobulinemia
• A Randomized, Double-Blind, Placebo-Controlled, Phase 3 Study of Ibrutinib or Placebo in Combination with Rituximab in Subjects with Waldenström’s Macroglobulinemia

• Pembrolizumab plus Chemotherapy in Metastatic Non–Small-Cell Lung Cancer

Expectedness Assessment: Expected / Unexpected Adverse Events and SUSAR

In clinical trials, reporting of adverse events is critical to ensure the safety of participants. When an adverse event is reported, it is also assessed for the severity, seriousness, causality, and outcome. All of these are the standard fields that are supposed to be collected on the case report forms and on serious adverse event (SAE) forms. 

Expectedness of an adverse event is also critical; however, the assessment of expectedness is usually not collected on the case report form or SAE form because the responsibility of the expectedness evaluation is not on investigator’s side, but on the sponsor’s side. SUSAR (suspected unexpected serious adverse reaction) must be reported to regulatory agencies and IRBs in expedited way.

Definition of Expectedness:

According to ICH E2A “CLINICAL SAFETY DATA MANAGEMENT: DEFINITIONS AND STANDARDS FOR EXPEDITED REPORTING”, the unexpected adverse drug reaction is defined as the following:

3. Unexpected Adverse Drug Reaction An adverse reaction, the nature or severity of which is not consistent with the applicable product information (e.g., Investigator's Brochure for an unapproved investigational medicinal product). (See section III.C.)

In each individual study protocol, the definition may be a little bit different, but essentially the same.

Unexpected: – Not listed in Investigator Brochure or is not listed at the specificity or severity that has been observed, or, if an investigator brochure is not required or available, is not consistent with the risk information described in the general investigational plan or elsewhere in the current application.

An unexpected adverse reaction has a nature or severity of which is not consistent with the study intervention description (e.g. Investigator's Brochure for an unapproved investigational product or package insert/summary of product characteristics for an approved product). The unexpected AE must be reported, whether related to the study intervention or not, with as much detail as is available

Expected: - listed in Investigator Brochure.

The Purpose of Expedited Reporting of the Suspected Unexpected Serious Adverse Events (SUSAR)

According to ICH E2A “CLINICAL SAFETY DATA MANAGEMENT: DEFINITIONS AND STANDARDS FOR EXPEDITED REPORTING E2A”, the purpose of the SUSAR reporting is defined as:

C. Expectedness of an Adverse Drug Reaction The purpose of expedited reporting is to make regulators, investigators, and other appropriate people aware of new, important information on serious reactions. Therefore, such reporting will generally involve events previously unobserved or undocumented, and a guideline is needed on how to define an event as "unexpected" or "expected" (expected/unexpected from the perspective of previously observed, not on the basis of what might be anticipated from the pharmacological properties of a medicinal product). As stated in the definition (II.A.3.), an "unexpected" adverse reaction is one, the nature or severity of which is not consistent with information in the relevant source document(s). Until source documents are amended, expedited reporting is required for additional occurrences of the reaction.

The following documents or circumstances will be used to determine whether an adverse event/reaction is expected: 1. For a medicinal product not yet approved for marketing in a country, a company's Investigator's Brochure will serve as the source document in that country. (See section III.F. and ICH Guideline for the Investigator's Brochure.) 2. Reports which add significant information on specificity or severity of a known, already documented serious ADR constitute unexpected events. For example, an event more specific or more severe than described in the Investigator's Brochure would be considered "unexpected". Specific examples would be (a) acute renal failure as a labeled ADR with a subsequent new report of interstitial nephritis and (b) hepatitis with a first report of fulminant hepatitis.

Expectedness/Unexpectedness Not Collected in Case Report Forms or SAE Forms

CDISC/CDASH “Clinical Data Acquisition Standards Harmonization (CDASH) User Guide” excluded the collection of ‘expected criteria’, citing that it is “handled in Clinical Investigative Brochure”



In FDA’s Guidance for Industry and Investigators “Safety Reporting Requirements for INDs and BA/BE Studies”, the reporting responsibility for unexpected adverse events is specified for sponsors (not the investigators). 

Two Types of Unexpectedness and Handling the List of Unexpected AEs

According to a blog post "Seriousness, Expectedness and Investigator Brochures", there are actually two types of unexpectedness 

The first is “regulatory expectedness”. This refers to the SAEs that the company considers likely/possibly or probably related to the study drug. This list is used to determine whether an SAE is a SUSAR (Suspected, unexpected serious adverse reaction) and thus expeditable to FDA, EMA and other health agencies.

The second we can call “clinical expectedness” which is a listing of SAEs that the investigator and patient may encounter during the trial and should be aware of. They may be due to the drug, the disease, comedications, concomitant illnesses (e.g. the flu) or other causes. These may or may not be the same as the “regulatory expectedness” list of SAEs but are important for the treating physician to be aware of and look for. It may not be possible yet to determine whether the particular SAE is due to the drug or the disease or comedications etc. This may become clearer later in the drug’s lifespan as more data becomes available; but sometimes it does not ever become clear.

Due to different understanding of the expectedness, different companies may act differently in handling the expectedness assessment. Some companies (especially the European companies) may want to add as many AEs as possible to the Investigator Brochure so that less AEs would meet the unexpected criteria – specifically the SUSAR criteria for expedite reporting.

Other companies may want to add as few AEs as possible to the Investigator Brochure because two many AEs listed in the Investigator Brochure would give the investigators an impression that the investigational product is not safe.

The right approach should be that the list of AEs/SAEs should be carefully reviewed by one or more medically qualified persons to decide if terms should be included (added) in the Investigator Brochure.

Expectedness assessment is more for fulfilling the sponsor’s reporting responsibility and expected / unexpected AEs are evaluated by the sponsor (not the investigators) through comparing to the Investigator Brochure or product label.

Expected AEs versus AEs of Special Interest (AESI)

Sometimes, the study protocol may include a list of AEs of Special Interest. According to FDA guidance for industry: E2F Development Safety Update Report AEs of Special Interest are defined as following.

“Adverse event of special interest: An adverse event of special interest (serious or non-serious) is one of scientific and medical concern specific to the sponsor’s product or program, for which ongoing monitoring and rapid communication by the investigator to the sponsor can be appropriate. Such an event might warrant further investigation in order to characterize and understand it. Depending on the nature of the event, rapid communication by the trial sponsor to other parties (e.g., regulators) might also be warranted. (Based on CIOMS VI)”

Here are some examples of AESI: distal emboli events in clinical trials using thrombolytic agents, syncope events in pulmonary arterial hypertension studies, diarrhea in Irritable Bowel Syndrome studies) . I had a previous post "Adverse Event of Special Interest (AESI), Standardized MedDRA Query (SMQs), Customer Queries (CQs), and SAS Programming"

AEs of Special Interest are usually the expected AEs.

An event can be an unexpected AE in the early development stage, but become the expected AE and AE of Special Interest in late stage. For example, during the TYSABRI® (natalizumab) (for multiple sclerosis) drug development, a rare brain infection—called progressive multifocal leukoencephalopathy (PML)— was unexpected in early stage, and then become an AE of Special Interest.

Grading the Severity of AEs and its Impact on AE Reporting

For all adverse events including serious adverse events in clinical trials, severity (or intensity) should be assessed and recorded. AE severity used to be called AE intensity. Nowadays, severity is more commonly used. The assessment of severity is based on the investigator’s clinical judgement, therefore, there are lot of subjective judgement in the AE severity assessment/reporting.

There seems to be three different grading scale in assessing/recording the severity:

Mild, Moderate, and Severe

This is commonly used in non-oncology studies. The definition of the mild, moderate, and severe may be different from one study protocol to another. The severity (intensity) of each AE including SAE recorded in the CRF should be assigned to one of the following categories:

• Mild: An event that is easily tolerated by the subject, causing minimal discomfort and not interfering with everyday activities.
• Moderate: An event that is sufficiently discomforting to interfere with normal everyday activities.
• Severe: An event that prevents normal everyday activities.

 or 

• Mild: awareness of sign or symptom, but easily tolerated
• Moderate: discomfort sufficient to cause interference with normal activities
• Severe: incapacitating, with inability to perform normal activities

CTCAE 

In oncology clinical trials, the AE severity is usually graded according to NCI’s AE Severity Grading Scale -  Common Terminology Criteria for Adverse Events (CTCAE). CTCAE can also be used to grade the AE for non-oncology studies, but generally not appropriate for studies using healthy volunteers.

• Grade 1 Mild; asymptomatic or mild symptoms; clinical or diagnostic observations only; no intervention indicated
• Grade 2 Moderate; minimal, local or noninvasive intervention indicated; limiting age-appropriate instrumental ADL
• Grade 3 Severe or medically significant but not immediately lifethreatening; hospitalization or prolongation of hospitalization indicated; disabling; limiting self care ADL
• Grade 4 Life-threatening consequences; urgent intervention indicated.
• Grade 5 Death related to AE.

Vaccine's Trials

In FDA’s guidance on vaccine trials “Toxicity GradingScale for Healthy Adult and Adolescent Volunteers Enrolled in PreventiveVaccine Clinical Trials”, the AE severity based on clinical abnormalities and laboratory abnormalities was graded as

• Mild (grade 1)
• Moderate (Grade 2)
• Severe (Grade 3)
• Potentially Life Threatening (Grade 4)

In statistical summaries, the grade 1 is counted as ‘mild’, the grade 2 as ‘moderate’, >= grade 3 will be counted as ‘severe’.  

During the course of an adverse event, the severity may change – which may have impact on how we report the adverse event.

In one of the previous posts ‘SAE Reconciliation and  Determining / recording the SAE Onset Date’, we discussed that an AE with the change in seriousness might need to be split into two events for recording: one non-serious AE with onset date of the first sign/symptom and one serious AE with onset date of the event meeting one of the SAE criteria. The similar issue arises when we try to record the AE with severity change.

The most common instruction for AE recording is that when there is severity change, a new AE should be recorded. Here are some example instructions:

Start Date
Record the date the adverse event started. The date should be recorded to the level of granularity known (e.g., year, year and month, complete date) and in the specified format. If a previously recorded AE worsens, a new record should be created with a new start date. There should be no AE start date prior to the date of the informed consent. Any AE that started prior to the informed consent date belongs instead in the medical history. If an item recorded on the medical history worsens during the study, the date of the worsening is entered as an AE with the start date as the date the condition worsened.

End Date
Record the date the adverse event stopped or worsened.  The date should be recorded to the level of granularity known (e.g., year, year and month, complete date) and in the specified format.  If an AE worsens, record an end date and create a new AE record with a new start date and severity. 

If the AE increases in severity per the DAIDS Grading Table, a new AE Log CRF should be completed to document this change in severity.

the eCRF Completion Guidelines for adverse events:  Enter a new event if action taken, seriousness, causality, severity (intensity), etc. changes over the course of an adverse event.  A timestamp for any changes in events can be seen in the data via event start/stop dates.

However, this way of recording the adverse events may result in splitting the single event into multiple adverse events and may result in over reporting in the number of adverse events.

Suppose the subject experienced a headache adverse event, the event started with mild intensity, then progressed to moderate, and then went back to the mild intensity. Should this headache be reported as three separate adverse events (two with mild severity and one with moderate severity)? or Should it be reported as single event with moderate severity?

This question was submitted to FDA and the FDA response (see the link below) suggested that this should be reported as one event (with the maximum severity)

AE Reporting Question - Changes in Severity (03/08/2010)

The second question and answer explicitly stated:

Question 2:

[Redacted] is the sponsor of the study. We have been advised by our data coordinating center to record an AE that changes in severity as two AEs instead of 1 AE - starting a new AE each time the severity changes. This convention is different than that of our previous coordinating center and has caused us great concern.

Answer 2:

We have concerns that an approach to adverse event reporting as you described below (i.e., a change in severity of an adverse event necessitates a new adverse event report) may inaccurately reflect the adverse event profile for the product. Therefore, we strongly recommend that you contact the FDA review division regulating this clinical investigation for additional input on the most scientifically and medically sound approach to the adverse event reporting specifically for this trial.

I recently submitted this same question to FDA’s OC GCPQuestions and Answers and got the following response:

Question: 

We constantly run into the issue how to record the adverse event in the database in the situation there is a severity change or seriousness change during the course of the adverse event.
 A subject in clinical trial reported a mild headache. Two days later, the headache became moderate in severity. Then headache became mild in severity again.
 In this case, shall we record this as one headache event with moderate severity or record as three headache events (a new event is record whenever there is a severity change)?
 Similarly, a subject in clinical trial reported a non-serious adverse event. Several days later, subject needs to be hospitalized for this adverse event – now the event meets the seriousness criteria.
 In a situation of a non-serious adverse event becoming serious, shall we record it as a single AE with seriousness or shall we record as two separate AEs (one non-serious AE and one serious AE)?

OC-GCP Response:

Given your brief description that the subject's headache is ongoing, it would seem that this adverse event would best be reported as a single event with variable severity. However, the clinical judgment of the principal investigator (or, if the principal investigator is not a clinician, then a physician consultant to the research) would be helpful in clarifying the symptoms and hence the reporting of the adverse event(s). There are several cogent clinical scenarios the understanding of which would require more information than you have supplied. For example, the subject's symptomatology could represent an unremitting headache of several days duration or episodic headaches of finite duration with varying intensities or a symptom of another event altogether such as a change in blood pressure, etc. The same would apply for the hospitalization event.
 To best sort out the adverse event(s) itself and therefore the appropriate reporting, I would recommend a clinical assessment of the headache. In addition, the protocol may have detailed how adverse events should be reported. As well, the sponsor (I'm not sure of [Redacted] status in this trial, i.e., is/is not the sponsor) may have specifications for adverse event reporting that could guide you. If you still feel uncertain, I would strongly recommend contacting the FDA review division regulating this trial.
 Lastly, if it becomes apparent that this same "fact pattern" recurs, it may be advisable for the sponsor to clearly articulate standards for adverse event reporting such that there can be consistency in reporting of headaches.

From the statistical analysis standpoint, whether or not it is recorded as one event with maximum severity or multiple events with various seventies do not have impact on our calculation of the incidence of AEs. However, it will have great impact on the calculation of the number of AEs.

It is the common understanding that if an event recorded on the medical history worsens during the study or after the initiation of the study drug, a new AE should be recorded and the date of the worsening is entered as the new AE onset date