Change from Baseline versus Percent Change from Baseline

The US Food and Drug Administration (FDA) has published its fourth and final guidance in a series of patient-focused drug development (PFDD) guidances meant to help sponsors collect and incorporate patient experience information that can factor into regulatory decision-making. The latest guidance "Patient-Focused Drug Development: Incorporating Clinical Outcome Assessments Into Endpoints For Regulatory Decision-Making" focuses on how clinical outcomes assessments (COA) can be used as endpoints to support a product. It is interesting to see that there is an entire section to discuss the difference between using change from baseline versus percent (or percentage) change from baseline as the endpoint. While the discussion is specifically for COA (clinical outcomes assessments) endpoints, the same discussion points are applicable to other endpoints where the outcome measures are continuous variables. 

Clinical trials are usually designed as longitudinal studies where the baseline measures are performed before the randomization or the first dose of the study drug and then there will be periodic measures (or repeated measures) for the post-baseline visits. 

The statistical analyses may be performed on the original measures, but are more often performed using change from baseline values. For each post-baseline visit, the change from baseline values will be calculated and statistical analyses will include the change from baseline values as the dependent variable and the baseline values will be used as a covariate in the model. Both FDA and EMA have guidelines related to the adjustment for baseline values. 

• FDA guidance (2021) "Adjusting for Covariates in Randomized ClinicalTrials for Drugs and Biological Products"

"Clinical trials often record a baseline measurement of a defined characteristic and record a later measurement of the characteristic to be used as an outcome. When using this approach, adjusting for the baseline value rather than (or in addition to) defining the primary endpoint as a change from baseline is generally acceptable."

• EMA guidelines (2015) "Guideline on adjustment for baseline covariates in clinical trials"

"5.6. Change from baseline’ analyses

When the primary analysis is based on a continuous outcome there is commonly the choice of whether to use the raw outcome variable or the change from baseline as the primary endpoint. Whichever of these endpoints is chosen, the baseline value should be included as a covariate in the primary analysis. The use of change from baseline with adjustment for baseline is generally more precise than change of baseline without adjustment. Note that when the baseline is included as a covariate in a standard linear model, the estimated treatment effects are identical for both ‘change from baseline’ (on an additive scale) and the ‘raw outcome’ analysis. Consequently if the appropriate adjustment is done, then the choice of endpoint becomes solely an issue of interpretability."

Percent change from baseline may also be used as the endpoint even though it is less commonly used in the analysis of clinical trial data. FDA's COA guidance clearly indicated the potential issues for using percent change from baseline in the analysis. 

For statistical modeling, Percent change from baseline has some undesirable properties: 

• It is asymmetric, e.g. a change from 4 to 5 is 25%, but a change from 5 to 4 is -20%. While this is asymmetric, the percent change has been commonly used in measuring the fluctuation of stock prices, a change from $50 to $100 is a 100% increase, but a change from $100 to $50 is a 50% decrease. 
• The variance is not easily defined. If we rewrite the percent change from baseline, it will be 1−Post-baseline measures/Baseline measure, so the variance is only defined by the ratio of the Post baseline values/baseline value. In statistical modeling, people don't like to deal with the ratio unless there is no better way. When we handle log-normal data (such as pharmacokinetic data), the geometric mean ratio is commonly used. 
• It is undefined if the baseline is zero

In a paper by Vickers "The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study", the following conclusions were made:

"Percentage change from baseline has the lowest statistical power and was highly sensitive to changes in variance. Theoretical considerations suggest that percentage change from baseline will also fail to protect from bias in the case of baseline imbalance and will lead to an excess of trials with non-normally distributed outcome data."

Nevertheless, the percent change from the baseline may still be used as the primary efficacy endpoint in some clinical trials. Recently, therapeutic trials for weight loss in non-diabetic patients are hot topics. In clinical trials for weight loss, the primary efficacy endpoint is the Percent Change from Baseline to Week x in weight or BMI (body mass index). 

In a paper by Weghuber, et al "Once-Weekly Semaglutide in Adolescents with Obesity", the primary efficacy endpoint was percent change from baseline to week 68 in BMI. 

Efficacy end points were assessed from baseline (the time of randomization [week 0]) to week 68, unless otherwise stated. The primary end point was the percentage change in BMI, and the secondary confirmatory end point was a reduction in body weight of

at least 5%.

In a paper by Rubino et al "Effect of Weekly Subcutaneous Semaglutide vs Daily Liraglutide on BodyWeight in Adults With Overweight or Obesity Without Diabetes The STEP 8 Randomized Clinical Trial", the primary efficacy endpoint was percentage change from baseline in body weight at week 68. Percent change from baseline was analyzed using analysis of covariance, with randomized treatment as a factor and baseline value of the outcome measure of interest (eg, baseline body weight in kilograms for analysis of percentage change in body weight) as a covariate. Multiple imputation approach was used to handle the missing values. 

In SURMOUNT-1 study by Eli Lilly (Jastreboff, et al "Tirzepatide Once Weekly for the Treatment of Obesity"), co-primary efficacy endpoints were specified: 

Percent change in body weight from baseline to Week 72

AND

Percentage of participants with ≥5% body weight reduction at Week 72

Obtaining standard deviations from standard errors and confidence intervals for group means

Sometimes, we need to obtain a standard deviation (SD) in order to calculate the sample size for a new clinical trial where the primary efficacy endpoint is continuous measure. However, when we look at the literature, the SD may not be presented. Instead, the standard error (SE) or confidence intervals (CIs) are presented. The SD can be obtained from the SE or CIs. 

Below are texts from Cochrane Handbooks:

A standard deviation can be obtained from the standard error of a mean by multiplying by the square root of the sample size:

When making this transformation, standard errors must be of means calculated from within an intervention group and not standard errors of the difference in means computed between intervention groups.

 

Confidence intervals for means can also be used to calculate standard deviations. Again, the following applies to confidence intervals for mean values calculated within an intervention group and not for estimates of differences between interventions (for these, see Section 7.7.3.3). Most confidence intervals are 95% confidence intervals. If the sample size is large (say bigger than 100 in each group), the 95% confidence interval is 3.92 standard errors wide (3.92 = 2 × 1.96). The standard deviation for each group is obtained by dividing the length of the confidence interval by 3.92, and then multiplying by the square root of the sample size:

For 90% confidence intervals 3.92 should be replaced by 3.29, and for 99% confidence intervals it should be replaced by 5.15.

 

If the sample size is small (say less than 60 in each group) then confidence intervals should have been calculated using a value from a t distribution. The numbers 3.92, 3.29 and 5.15 need to be replaced with slightly larger numbers specific to the t distribution, which can be obtained from tables of the t distribution with degrees of freedom equal to the group sample size minus 1. Relevant details of the t distribution are available as appendices of many statistical textbooks, or using standard computer spreadsheet packages. For example the t value for a 95% confidence interval from a sample size of 25 can be obtained by typing =tinv(1-0.95,25-1) in a cell in a Microsoft Excel spreadsheet (the result is 2.0639). The divisor, 3.92, in the formula above would be replaced by 2 × 2.0639 = 4.128.

 

For moderate sample sizes (say between 60 and 100 in each group), either a t distribution or a standard normal distribution may have been used. Review authors should look for evidence of which one, and might use a t distribution if in doubt.

 

As an example, consider data presented as follows:

Group	Sample size	Mean	95% CI
Experimental intervention	25	32.1	(30.0, 34.2)
Control intervention	22	28.3	(26.5, 30.1)

The confidence intervals should have been based on t distributions with 24 and 21 degrees of freedom respectively. The divisor for the experimental intervention group is 4.128, from above. The standard deviation for this group is √25 × (34.2 – 30.0)/4.128 = 5.09. Calculations for the control group are performed in a similar way.

 

It is important to check that the confidence interval is symmetrical about the mean (the distance between the lower limit and the mean is the same as the distance between the mean and the upper limit). If this is not the case, the confidence interval may have been calculated on transformed values (see Section 7.7.3.4).

In the literature, the SEs and CIs are usually calculated from more sophisticated models (analysis of covariance, mixed model,...) - analyses adjusted for additional covariates. The methods described above can still be used to obtain the SDs - the calculated SDs should still be provide a good approximation of the SDs that are needed for planning future trials. 

in a paper by Jastreboff et al (2022) "Tirzepatide Once Weekly for the Treatment of Obesity", the sample size calculation was based on group mean difference and common standard deviation. 

"We calculated that a sample size of 2400 participants would provide an effective power of greater than 90% to demonstrate the superiority of tirzepatide (10 mg, 15 mg, or both) to placebo, relative to the coprimary end points, each at a two-sided significance level of 0.025. The sample-size calculation assumed at least an 11-percentage-point difference in the mean percentage weight reduction from baseline at 72 weeks for tirzepatide (10 mg, 15 mg, or both) as compared with placebo, a common standard deviation of 10%, and a dropout rate of 25%."

However, the study results were presented with LS mean difference in percentage change in body weight between two groups and their 95% confidence intervals. Standard deviations were not provided, but can be easily calculated using the method described above. 

Using Tirzepatide 15 mg group as an example, SD for 'percent change in body weight' can be calculated as SD1 =sqrt(630) * (-19.9 - (-21.8))/3.92 = 12.2; SD for 'Difference from placebo in percentage change in body weight' can be calculated as sqrt(630) x (-16.3 - (-19.3))/3.92 =19.2. The actual SDs from the study data are a little bit higher than the assumed SD of 10%.

 

"Ensuring Public Trust in an Empowered FDA" - Accelerated Approval

 In the latest issue of New England Journal of Medicine, Drs Ross, Berg, and Ramachandran wrote a paper:

"Ensuring Public Trust in an Empowered FDA". 

The paper discussed the recent trend that FDA is granting drug approval using the accelerated approval pathway - approval based on the biomarker or surrogate end points that are deemed “reasonably likely” to predict clinical benefit. 

The accelerated approval pathway was one of the expedited pathways for drug approval in FDA's guidance "Expedited Programs for Serious Conditions – Drugs and Biologics". The guidance defined the accelerated approval as the following: 

Accelerated approval has been historically used in HIV drug approval and oncology drug approval (mostly successful), however, in recent years, the FDA has increasingly expanded the use of the accelerated approval program beyond HIV and oncology therapies - mainly in the neurology area such as DMD, Alzheimer's disease, and ALS. 

The FDA argued that the FDA should exercise “the greatest flexibility possible” under its statutory authority in considering accelerated approval for drugs for the treatment of serious conditions with unmet medical needs. However, applying "the greatest flexibility possible" can empower the FDA, but it may come with consequences. 

Accelerated approval is based on a surrogate endpoint that is reasonably likely to predict clinical benefits, however, the "reasonably likely to predict clinical benefits" can be difficult to prove, and a lot of time is claimed based on conflicting evidence. According to the paper by Drs Fleming and Powers "Biomarkers and Surrogate Endpoints In Clinical Trials", it is extremely difficult to verify a biomarker can be a surrogate endpoint and can reasonably likely predict clinical benefits. 

The authors concluded:

 "An empowered FDA may prioritize using its regulatory authority to enable timely access to innovative products, but to ensure continued trust in the agency, this priority should be balanced against the challenges clinicians, patients, and caregivers face when there is substantial residual uncertainty about product safety and efficacy."

Also noted was that the FDA established a website "Accelerated Approval Program" where drugs approved through accelerated approval pathway were listed and the status of confirmatory trials to prove the clinical benefit was also listed. 'Ongoing' indicates that the confirmatory trial is ongoing; 'verified clinical benefit indicates the clinical benefit has been confirmed; and 'withdrawn' indicates that the clinical benefit has not been confirmed in the confirmatory trials and the drug approved through accelerated approval has been withdrawn. 

Accelerated Approval is a hot topic lately: 

• FDA Issues Draft Guidance Aimed at Improving Oncology Clinical Trials for Accelerated Approval
• Clinical Trial Considerations to Support Accelerated Approval of Oncology Therapeutics
• Top FDA official: Agency needs to start using accelerated approval for gene therapies
• US FDA to take steps to help gene therapies get accelerated approval

The condition for FDA to grant the accelerated approval is that the sponsor is required to do a confirmatory study (either already ongoing or will start right after the accelerated approval) to verify the drug's clinical benefits. A recent article in JAMA titled "Timing of Confirmatory Trials for Drugs Granted Accelerated Approval Based on Surrogate Measures From 2012 to 2021" concluded the following:

"Accelerated approval expedites the marketing of medications based on uncertain efficacy evidence, but the process depends on timely follow-up trials.We found that more than half of so-called confirmatory studies were not completed in the agreed-on time. In contrast with a recent Office of the Inspector General Report of incomplete confirmatory trials, our study includes late completed trials and manufacturer-reported delays. Limitations include shorter follow-up for more recent accelerated approvals and inability to identify reasons for trial delays.

Incomplete confirmatory clinical trials harm patients who are prescribed expensive drugs

despite uncertain clinical benefits. However, drug manufacturers face few consequences for delays. The Consolidated Appropriations Act for 20236 included accelerated approval reforms, such as granting the FDA greater authority to ensure confirmatory trials are under way before approval, mandating progress reports every 6 months by manufacturers, and clarifying procedures for withdrawal if follow-up trials do not find clinical benefit. It will be important to monitor whether these changes lead to fewer delays or whether additional authority is needed to assure that confirmatory trials are completed in a timely manner for the benefit of patients."