Real world data (RWD) and Real world evidence (RWE) in Drug Development

The 21st Century Cures Act (Cures Act), signed into law on December 13, 2016, is designed to accelerate medical product development and bring new innovations and advances faster and more efficiently to the patients who need them. Following the passing of the Cures Act, the Food and Drug Administration (FDA) has created a framework for evaluating the potential use of real-world evidence (RWE) to help support the approval of a new indication for a drug already approved or to help support or satisfy drug postapproval study requirements.

In December, 2018, FDA issued "Framework for FDA’s Real-World Evidence Program" and FDA's CDER and CBER divisions (now also including the oncology center of excellence) created the RWE program. A series of guidance documents were released. 

DEFINITION of RWD and RWE:

FDA GUIDANCE DOCUMENTS on RWD and RWE (as of November 2024):

Topic	Title	Category	Current Status
EHRs and claims data	Real-World Data: Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products	Data considerations	Final, July 2024
Registry data	Real-World Data Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products	Data considerations	Final, December 2023
Data standards	Data Standards for Drug and Biological Product Submissions Containing Real-World Data	Data submission	Final, December 2023
Regulatory considerations	Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products	Applicability of regulations	Final , August  2023
Submitting RWE	Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drug and Biological Products	Procedural	Final, September 2022
Externally controlled trials	Considerations for Design and Conduct of Externally Controlled Trials for Drug and Biological Products	Design considerations	Draft, February 2023
Non-interventional studies	Considerations Regarding Non-Interventional Studies for Drug and Biological Products	Design considerations	Draft, March 2024
RCTs in clinical practice settings	Integrating Randomized Controlled Trials for Drug and Biological Products Into Routine Clinical Practice	Design considerations	Draft, September 2024

WEBINARS for RWD/RWE:

FDA officials have given various webinars to explain these RWD/RWE guidance documents and encourage the sponsors to apply the RWE to the drug approval process. A non-profit organization, the Reagan-Udall Foundation for the FDA, in collaboration with the Food and Drug Administration (FDA), hosted a series of free, public webinars to discuss FDA-issued guidance in the RWD/RWE. 

Title	Webinar Series	Date
Real-World Data: Assessing Electronic Health Records and Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products	https://reaganudall.org/news-and-events/events/public-webinar-series-fda-issued-guidance-real-world-evidence	November 4, 2021
Data Standards for Drug and Biological Product Submissions Containing Real-World Data	https://reaganudall.org/news-and-events/events/real-world-data-webinar-series-data-standards	December 3, 2021
Real-World Data Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products	https://reaganudall.org/news-and-events/events/real-world-data-webinar-series-registries	January 28, 2022
Considerations for the Use of Real-World Data and Real-World Evidence to Support Regulatory Decision-Making for Drug and Biological Products	https://reaganudall.org/news-and-events/events/real-world-data-webinar-series-considerations-use-rwd-and-rwe	February 11, 2022
Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drug and Biological Products	No webinar was conducted
Considerations for Design and Conduct of Externally Controlled Trials for Drug and Biological Products	https://www.youtube.com/watch?v=5rfInDy7osw&t=1s	April 13, 2023
Considerations Regarding Non-Interventional Studies for Drug and Biological Products	https://reaganudall.org/news-and-events/events/real-world-evidence-webinar-series-considerations-regarding-non	May 30, 2024
Integrating Randomized Controlled Trials for Drug and Biological Products Into Routine Clinical Practice	https://reaganudall.org/news-and-events/events/real-world-evidence-webinar-series-integrating-randomized-controlled-trials https://youtu.be/VRaQyOvn3AM?si=YrM9pY6JhL3LBr_o	November 22, 2024

Duke Margolis Center for Health Policy, in collaboration with the FDA, also conducted a series of free, public webinars to discuss the application of RWD/RWE: 

Webinar Title/Link	Date
Optimizing the Use of Real-World Evidence in Regulatory Decision-Making for Drugs and Biological Products – Looking Forward	December 12, 2024
2024 State of Real-World Evidence Policy	July 25, 2024
The State of Real-World Evidence Policy 2023	September 28, 2023
Understanding the Use of Negative Controls to Assess the Validity of Non-Interventional Studies of Treatment Using Real-World Evidence	March 8, 2023
Workshop on Draft Guidance on Real-World Data: Electronic Health Records/Medical Claims Data and Data Standards	February 27, 2023
The State of Real-World Evidence Policy	May 12, 2022
An Introduction to Real-World Data & Real-World Evidence: A Virtual Training Series for the Patient Community	March 12, 2021

SUMMARY:

RWD / RWE play an increasingly vital role in drug development by complementing traditional clinical trial data. Derived from sources such as electronic health records, insurance claims, registries, and patient-reported outcomes, RWD provides insights into how drugs perform in diverse, routine care settings. RWE, generated by analyzing RWD, helps assess the safety, efficacy, and value of treatments in real-world populations, addressing gaps that controlled clinical trials may leave. These insights are particularly valuable in identifying long-term outcomes, supporting regulatory decisions, designing pragmatic trials and comparative effectiveness researches, and informing post-market safety surveillance. Regulatory agencies like the FDA and EMA are encouraging the integration of RWE to enhance decision-making, optimize study designs, and support label expansions or accelerated approvals.

The Use of External Controls in FDA Regulatory Decision Making and Bayesian Borrowing

An good article by Jahanshahi et al "The Use of External Controls in FDA Regulatory Decision Making". The authors reviewed and summarized FDA regulatory approval decisions between 2000 and 2019 for drug and biologic products and summarized the pivotal studies that leveraged external controls, with a focus on select therapeutic areas.

The paper is open-access and available at "The Use of External Controls in FDA Regulatory Decision Making". 

The leading author is from BioMarin Pharmaceutical whose biological product Brineura for treatment of the rare Batten disease was approved by FDA based on an single arm study leveraging the external control from a historical registry. 

In the latest issue of the New England Journal of Medicine, Richeldi et al published a paper "Trial of a Preferential Phosphodiesterase 4B Inhibitor for Idiopathic Pulmonary Fibrosis". The trial was designed as a smaller (in sample size) trial with 2:1 randomization ratio by leveraging the data from the Placebo control groups in historical clinical trials. Bayesian borrowing (or Bayesian dynamic borrowing) approach was used for the analyses, as depicted below. While this is an example of successful use of external control in phase 2 study, it is unlikely that the same approach can be employed in their phase 3 pivotal studies mainly because that idiopathic pulmonary hypertension is a rare disease, but not rare enough. 

Synthetic Control Arm (SCA), External Control, Historical Control

Lately, the term 'synthetic control' or 'synthetic control arm' or SCA, in short, is becoming popular - it is mainly driven by the desire to design more efficient clinical trials that are not traditional, the golden standard RCT (randomized controlled trials) with a concurrent control group. 

In a previous post, I compared historical control versus external control in clinical trials. The subtle difference is mainly in the time element. Historical control is one type of external control, but the reverse is not true. External control can be historical control or contemporaneous control. For example, in a clinical trial to assess the efficacy and safety of the donor lung preserved using ex-vivo lung perfusion (EVLP) technique, the EVLP lung transplantation cohort was compared to a contemporaneous (not concurrent) control cohort that was formed through the matched control from the traditional lung transplantation patients.   

Then what is 'synthetic control' or 'synthetic control arm'?

Synthetic control arm is the use of synthetic data as a control arm in clinical trials. According to an article "Synthetic data in the civil service" in the latest issue of SIGNIFICANCE, synthetic data is defined as "artificially generated data that are modelled on real data, with the same structure and properties as the original data, except that they do not contain any real or specific information about individuals. The goal of synthetic data generation is to create a realistic copy of the real data set, carefully maintaining the nuances of the original data, but without compromising important pieces of personal information."

Synthetic control arm is a control arm generated through existing data resources representing normal patient statistics. Synthetic control arm can serve as a comparator for a single-arm clinical trial or augment the smaller concurrent control group (for example with active:control ratio of 3:1 or 4:1) in RCTs. 

In a presentation by at Harvard Medical School Executive Education Webinar Series,  Mr. Chatterjee presented "Synthetic Control Arms in Clinical Trials and Regulatory Applications" and he defined the 'synthetic control arm' as the following:

In a paper by Thorlund et al "Synthetic and External Controls in Clinical Trials – A Primer for Researchers", they stated that synthetic control arms are external control arms - two terms can be used interchangeably:

External control arms are also called “synthetic” control arms as they are not part of the original concurrent patient sample that would have been randomized into the experimental or the control treatment arms as in a traditional RCT. External controls can take many forms. For example, external control arms can be established using aggregated or pooled data from placebo/control arms in completed RCTs or using RWD (Real World Data) and pharmacoepidemiological methods. Pooled data from historical RCTs can serve as external controls depending on the availability of selected “must have” data, similarity of patients, recency and relevancy of experimental treatments that were tested, availability and similarity of relevant endpoints (eg, operational definitions and assessments), and similarity of other important study procedures that were conducted in these historical trials. It is important to note that using control data from historical RCTs still results in a nonrandomized comparison but has the advantage of standardized data collection in a trial setting and patients who enroll in clinical trials may have more similar characteristics than those who do not.

However, I think that there are subtle differences between these two terms. With 'synthetic' control arms, the term 'synthetic' implies there are some selection, manipulation, derivation, matching, pooling, borrowing from the source data. Just like the meta-analysis is also called research synthesis and requires the statistical approaches to combine the results from multiple scientific studies, the 'synthetic' control also requires the use of statistical approaches to process the data from multiple sources to form a control group to replace the concurrent control in traditional RCT clinical trials. 

The source data for constructing synthetic control can be the data from previous RCT clinical trials, real-world data, registry data, data from natural history studies, electronic health records, ... The source data must be the subject-level data, not the summary or aggregate data. 

ICH E10 "CHOICE OF CONTROL GROUP AND RELATED ISSUES IN CLINICAL TRIALS" included "External Control (including Historical Control)" as one of the options as the control groups in clinical trials. The external control here is not the same as synthetic control. 

1.3.5 External Control (Including Historical Control)

An externally controlled trial compares a group of subjects receiving the test treatment with a group of patients external to the study, rather than to an internal control group consisting of patients from the same population assigned to a different treatment. The external control can be a group of patients treated at an earlier time (historical control) or a group treated during the same time period but in another setting. The external control may be defined (a specific group of patients) or nondefined (a comparator group based on general medical knowledge of outcome). Use of this latter comparator is particularly treacherous (such trials are usually considered uncontrolled) because general impressions are so often inaccurate. So-called baseline controlled studies, in which subjects' status on therapy is compared with status before therapy (e.g., blood pressure, tumor size), have no internal control and are thus uncontrolled or externally controlled.  

How to Create a Synthetic Control Arm? 

The first step of creating a synthetic control arm is to harmonize the source data. The data from different sources or from different clinical trials should be standardized so that they can be used for the synthesis process. 

Various statistical approaches can be used to create a synthetic control arm. In an audiobook on synthetic control arms by Cytel, propensity scoring and Bayesian Dynamic Borrowing methods were discussed. 

The synthetic control arm can be considered as an approach of 'borrowing control' - i.e., some controls are borrowed from historical data. There are numerous options for borrowing controls: 

• Pooling: adds historical controls to randomized controls 
• Performance criterion: uses historical data to define performance criterion for current, treated-only trial to beat 
• Test then pool: test if controls sufficiently similar for pooling 
• Power priors: historical control discounted when added to randomized controls
• Hierarchical modeling: variation between current vs. historical data is modeled in Bayesian fashion 

In the article by Thorlund et al, the pros and cons of different methods for generating synthetic control arms were discussed. 

In Mr Chatterjee presentation, "Synthetic Control Arms in Clinical Trials and Regulatory Applications", there is a diagram to describe the process for creating a synthetic control arm. 

Even though the synthetic control arms, the use of real-world data, conducting the single-arm clinical trials are very appealing, the challenges are ahead and the regulatory acceptance is uncertain. There may be limited use in special cases (such as ultra-rare diseases, pediatric clinical trials) and for post-marketing activities (such as label expansion, label modification, post-marketing studies), but not in prime time to replace the concurrent control in traditional RCTs. 

In an article at Statnews.com "Synthetic control arms can save time and money in clinical trials", 

Even with the FDA making the use of real-world data a strategic priority, synthetic control arms can’t be used across the board to replace control arms. Synthetic control arms require that the disease is predictable (think idiopathic pulmonary fibrosis) and that its standard of care is well-defined and stable. That certainly isn’t the case for every disease.

It’s also important to consider that even when information is available from real-world data sources, it may be difficult to extract or of low quality. Routinely captured health care data, such as electronic health records, are typically siloed, fragmented, and unstructured. They are also often incomplete and difficult to access. New tools and methodologies are needed to consolidate, organize, and structure real-world data to generate research-grade evidence and ensure that confounding variables are accounted for in analyses. Analytic techniques such as natural language processing and machine learning will be needed to extract relevant information from structured and unstructured data.

The same view is also expressed in a Pink Sheet article "External Control Arms: Better Than Single-Arm Studies But No Replacement For Randomization".

Synthetic control group derived from historical clinical trial data could augment smaller randomized trials and yield better information than single-arm studies, but this approach should not be viewed as a substitute for randomized trials where feasible

ADDITIONAL REFERENCES:

• Alberto Abadie (2020) Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects
• Cytel Ebook (2020) Demystifying synthetic control arms
• Carrigan et al (2019) Using Electronic Health Records to Derive Control Arms for Early Phase Single-Arm Lung Cancer Trials: Proof-of-Concept in Randomized Controlled Trials
• Lim et al (2018) Minimizing Patient Burden Through the Use of Historical Subject-Level Data in Innovative Confirmatory Clinical Trials: Review of Methods and Opportunities
• Burcu et al (2019) Real‐world evidence to support regulatory decision‐making for medicines: Considerations for external control arms
• Ghadessi et al (2020) A roadmap to using historical controls in clinical trials – by Drug Information Association Adaptive Design Scientific Working Group (DIA-ADSWG)
• FDA-AACR Workshop: Real-world Evidence (July 29, 2019) Slide Decks
• Elizabeth Stuart The use of synthetic control and other covariate adjustment strategies for policy evaluation 
• Synth: An R Package for Synthetic Control Methods in Comparative Case Studies or Synth Package
• Exploring whether a synthetic control arm can be derived from historical clinical trials that match baseline characteristics and overall survival outcome of a randomized control arm: Case study in NSCLC
• Berry et al (2017) Creating a Synthetic Control Arm (SCA) from Previous Clinical Trials: Applications to Establishing Early Endpoints as Indicators of Overall Survival in Relapsed/Refractory Acute Myeloid Leukemia - poster  and Abstract

Real World Evidence in Regulatory Submissions - Story of IBRANCE for male breast cancer

In April 2019, FDA approved Pfizer's IBRANCE for male breast cancer. The approval is for the label extension for an already produce based on the real-world data (RWD). The IBRANCE approval in male breast cancer patients is considered as the first case that an FDA approval is mainly based on the RWD and has been used as an example in many lectures discussing the real-world evidence (RDE). Here is what is said in the press release about IBRANCE's approval for male breast cancer. 

“Men with breast cancer have limited treatment options, making access to medicines such as IBRANCE critically important,” said Bret Miller, founder of the Male Breast Cancer Coalition. “We applaud the use of real-world data, a new approach to drug review, to make IBRANCE available to certain men with metastatic breast cancer and help address an unmet need for these patients.”

Real-world data is playing an increasingly important role in expanding the use of already approved innovative medicines. Due to the rarity of breast cancer in males, fewer clinical trials are conducted that include men resulting in fewer approved treatment options. In the U.S. in 2019, it is estimated that there will be 2,670 new cases of invasive breast cancer and about 500 deaths from metastatic breast cancer in males. The 21st Century Cures Act, enacted in 2016, was created to help accelerate medical product development, allowing new innovations and advances to become available to patients who need them faster and more efficiently. This law places additional focus on the use of real-world data to support regulatory decision-making.

Detailed analysis of the use of IBRANCE in men with HR+, HER2- advanced or metastatic breast cancer will be presented at an upcoming medical meeting. Based on limited data from postmarketing reports and electronic health records, the safety profile for men treated with IBRANCE is consistent with the safety profile in women treated with IBRANCE.

In the original approval of IBRANCE for breast cancer, a randomized, controlled clinical trial was conducted, however, the male breast cancer patients were excluded from the study. In order to obtain the label extension to the male breast cancer patients, the RWE from electronic health records (EHR) for the off-label use of IBRANCE in male breast cancer patients was used. According to the FDA's review documents:

Male patients with breast cancer were ineligible in studies that provided the data to demonstrate the clinical benefit to support prior approvals of palbociclib (IBRANCE®). According to the current clinical practice standards, in the absence of safety and efficacy data from adequate and well-controlled studies, male patients with breast cancer are treated similarly to women with breast cancer. In this submission, the applicant provided the results of an analysis of real-world data (RWD) from electronic health records (EHRs) as additional supportive data to characterize the use of palbociclib in combination with endocrine therapy (aromatase inhibitor or fulvestrant) in male patients with breast cancer based on observed tumor responses in this rare subset of patients with breast cancer.

In a recorded webinar "Applying Real-World Evidence to Regulatory and Drug Development Challenges", Dr. Rebecca Miksad from Flatiron Health (the CRO who did the RWD analyses for Pfizer) summarized five key learnings from IBRANCE example for RWE in regulatory submissions: 

 

Pre-specification of study protocol & analysis plans 

“Without having reviewed and consented to a protocol and SAP, FDA cannot be certain that the protocol and SAP were pre-specified and unchanged during the data selection and analyses” 

- ODAC Briefing Document 

Appropriate Cohort selection for the research question 

Real-world patient cohorts need to be representative of population of interest 

Appropriate cohort selection criteria is context- and disease-dependent 

It is important to understand the feasibility of capturing each clinically meaningful variable 

A documented and traceable selection processes is needed (e.g., detailed cohort diagram) 

Missingness impacts ability of RWD to align with typical trial inclusion/exclusion (I/E) criterion 

Agreement is an additional layer of quality needs to be assessed for abstracted data 

Suitability of real-world endpoints 

Data quality suitability for the use case is critical for drawing confident conclusions from real-world endpoints 

depending on context, data quality considerations for real-world endpoints include: 

Can the endpoint be benchmarked to a reference or gold standard? 

How reproducible is the variable’s performance? 

Traceability back to source data 

Fit for purpose analytical methodologies 

Good analysis cannot fix low quality data; but bad analysis wastes high quality data 

Potential RWD quality issues need to be considered as part of the analysis plan. For example, 

Address potential bias in the data due to data quality issues (selection/ascertainment/confounding/immortal time) 

Assess the impact of missing data (sensitivity analysis) 

The cancerletter.com published an article including the responses from the regulatory agency (FDA), the sponsor (Pfizer), and the CRO (Flatiron Health) "How real-world evidence was used to support approval of Ibrance for male breast cancer". Some of the responses are copied here:

The Cancer Letter:

Was this the first approval based at least in part on real world evidence in oncology?

FDA:

Ibrance (palbociclib) was initially approved in 2015. It is a kinase inhibitor, now approved in combination with an aromatase inhibitor as the first hormonal-based therapy in women who have gone through menopause and in men, or with fulvestrant in patients whose disease progressed following hormonal therapy.

Pfizer provided the results of an analysis of real world data (RWD) from electronic health records (EHRs) as additional supportive data to characterize the use of Ibrance in combination with endocrine therapy (aromatase inhibitor or fulvestrant) in male patients with breast cancer based on observed tumor responses in this rare subset of patients with breast cancer.

Leveraging RWD to improve regulatory decisions is a key strategic priority for the FDA. This data may be derived from a variety of sources, such as electronic health records, medical claims, product and disease registries, laboratory test results and even cutting-edge technology paired with mobile devices.

These types of data are being used to develop real world evidence (RWE) that can better inform regulatory decisions.

Because they include data covering the experience of physicians and patients with the actual use of new treatments in practice, and not just in research studies, the collective evaluation of these data sources has the potential to inform clinical decision-making by patients and providers, develop new hypotheses for further testing of new products to drive continued innovation and inform us about the performance of medical products.

FDA has previously accepted RWD to support drug product approvals, primarily in the setting of oncology and rare diseases.

RWD has been used to determine prognosis or natural history of disease in order to help inform regulatory decision-making, for example, data on historical response rates drawn from expanded access, practice settings, or chart reviews.

 

The Cancer Letter:

What were the RWE endpoints being used here?

FDA:

The RWE endpoints used were real world tumor response and safety data. Real world tumor response was taken from the electronic health record as part of routine clinical care and information about each response event was retrospectively collected.

Therefore, this response included several factors, such as physical exam, symptom improvement, and pathology reports, which were used to supplement descriptions of radiology findings in the overall clinicians’ assessment of response.

Additional data on use and durations of prescriptions were also provided.

Pfizer:

The expanded indication in breast cancer is based on limited data from post-marketing reports and electronic health records sourced from three databases: IQVIA Insurance database, Flatiron Health Breast Cancer database and the Pfizer global safety database.

Based on these limited data, the safety profile for men treated with IBRANCE is consistent with the safety profile in women treated with IBRANCE.

A detailed analysis of the use of IBRANCE in men with HR+, HER2- advanced or metastatic breast cancer will be presented at an upcoming medical meeting.

Flatiron:

For this dataset, Pfizer engaged Flatiron to explore baseline characteristics, treatment patterns and clinical outcomes from patient-level, de-identified data for a group of male patients with metastatic breast cancer.

As with any project in which a partner is considering the inclusion of RWE as part of a regulatory submission, we consider it critical to ensure the data is “fit-for-purpose,” that is, ensuring that the dataset is fit for the intended use and can provide adequate scientific evidence.

Using RWE/RWD to support regulatory submission is a trend. Statisticians are meeting the challenges in developing the methods of integrating the RWD into the clinical trials and into the regulatory submissions. We are seeing the popular terms 'historical control', 'external control', 'synthetic control arms', 'digital twin', 'propensity score', 'Bayesian dynamic borrowing', 'causal inferences'... We hope to see that the regulatory agencies will be more receptive to the RWE in supporting regulatory approvals.