Cox proportional hazards regression model: univariate, multivariate, adjusted, stratified

Cox proportional hazards regression model has been called different names (Cox model, Cox regression model, Proportional hazards model, ... can be used interchangeably). The original paper by D.R. Cox "Regression models and life tables" is one of the most cited papers. Paired with the Kaplan-Meier method (and the log-rank test), the Cox proportional hazards model is the cornerstone for the survival analyses or all analyses with time to event endpoints.

With Cox regression model, we are now able to analyze the time to event data just like the linear regressions and logistic regressions to compare the treatment difference and to investigate the explanatory variables. The explanatory variables may be called independent variables, covariates,  confounding factors,...

Depending on whether the explanatory variables are continuous or categorical and depending on how the explanatory variables are used in the Cox regression model (as covariates, as stratification factors), Cox regression model can be fitted in different ways:

Univariate Cox regression	Unstratified and unadjusted
Multivariate Cox regression	Unstratified and adjusted
Stratified Cox regression	Stratified and unadjusted
Stratified Multivariate Cox regression	Stratified and adjusted

The variables used in adjusted Cox regression can be categorical or continuous, but the variables used in stratified Cox regression should be categorical.

The following are compiled from various sources listed below:

• What is a Cox model?
• Introduction to Cox Model
• Introduction to survival analysis in SAS
• SAS/STAT® 15.1 User’sGuide The PHREG Procedure
• Cox Proportional Hazards Model Using SAS 
• Statistical Analysis Plan for a NEJM article
• Cox proportional hazards model by STHDA
• A plain man's guide to the proportional hazards model

Univariate Cox Proportional Hazards Regression Model (Unstratified Unadjusted Proportional Hazards Regression Model)

The unstratified, unadjusted proportional hazards regression model is more commonly called Univariate Cox proportional hazards regression model and its assumptions are illustrated by:

h (.) hazard function as a function of time (relative to the start date), the patient’s

treatment and the unknown regression parameter

h0(t) unspecified baseline hazard function at time t

x_j1 treatment for patient j coded as 0 (placebo group) or 1 (treatment group); this coding leads to a hazard ratio less than 1 if the estimated effect suggests a lower hazard in the treatment group relative to the placebo group

β₁ unknown regression parameter for the treatment effect on the log-scale (the log hazard ratio).

The hazard ratio (HR) is a multiplicative constant λ1 = exp(β₁) comparing the hazard function in the treatment group relative to the hazard function in the placebo group (identical to the baseline hazard function). A HR less than 1 indicates decreased hazard for the event of interest in the treatment group compared to the placebo group.

SAS PROC PHREG (with the TIES=EXACT option for the “exact” handling of ties) is used to estimate the hazard ratio based on the partial maximum likelihood function; a Wald-test based two-sided CI is requested by the RISKLIMITS option (additional options controlling the output may be added):

PROC PHREG DATA=dataset;
MODEL tte*cnsr(1)=treat / TIES=EXACT RISKLIMITS ALPHA=;
RUN;

* tte represents variable containing event/censoring times;
* cnsr represents censoring variable (0=event, 1=censored);
* treat represents treatment group variable (0=placebo, 1=treatment);

Stratified (Unadjusted) Proportional Hazards Regression

The stratified unadjusted proportional hazards regression model and its assumptions are described by:

h (.) hazard function as a function of time (relative to the start date), the patient’s

stratum and treatment and the unknown regression parameter

h0k(t) unspecified baseline hazard function for stratum k at time t

xjk1 treatment for patient j in stratum k coded as 0 (placebo group) or 1 (treatment group); this coding leads to a hazard ratio less than 1 if the estimated effect suggests a lower hazard in the treatment group relative to the placebo group

β₁ unknown regression parameter for the treatment effect on the log-scale (the log hazard ratio).

The hazard ratio is a multiplicative constant λ1 = exp(β₁) comparing the hazard functions in the treatment group relative to the hazard functions in the placebo group. The latter are allowed varying across strata. However, the hazard ratio is assumed to be common across strata. An HR less than 1 indicates decreased hazard for the event of interest in the treatment group compared to the placebo group.

SAS PROC PHREG (with the TIES=EXACT option for the “exact” handling of ties) is

used to estimate the hazard ratio based on the partial maximum likelihood function; a Wald-test based two-sided CI is requested by the RISKLIMITS option (additional options controlling the output may be added):

PROC PHREG DATA=dataset;
MODEL tte*cnsr(1)=treat / TIES=EXACT RISKLIMITS ALPHA=;
STRATA strat1 .. stratj;
RUN;

* tte represents variable containing event/censoring times;
* cnsr represents censoring variable (0=event, 1=censored);
* treat represents treatment group variable (0=placebo, 1=treatment);
* strat1 to stratj represent stratification variables;

Adjusted Cox Proportional Hazards Regression Model (including Univariate Cox Proportional Hazards Regression Model and Multivariate Cox Proportional Hazards Regression Model)

The purpose of the model is to evaluate the effect of a single factor (univariate) or simultaneously the effect of several factors (multivariate) on survival. In other words, it allows us to examine how specified factor(s) influence the rate of a particular event happening (e.g., infection, death) at a particular point in time. This rate is commonly referred as the hazard rate. Predictor variable(s) (or factor(s)) are usually termed covariates in the survival-analysis literature.

The Cox model is expressed by the hazard function denoted by h(t). Briefly, the hazard function can be interpreted as the risk of dying at time t. It can be estimated as follow:

where,

·         t represents the survival time

·         h(t) is the hazard function determined by a set of p covariates (x₁,x₂,...,x_p)

·         the coefficients (b₁,b₂,...,b_p) measure the impact (i.e., the effect size) of covariates.

·         the term h₀ is called the baseline hazard. It corresponds to the value of the hazard if all the x_i are equal to zero (the quantity exp(0) equals 1). The ‘t’ in h(t) reminds us that the hazard may vary over time.

The Cox model can be written as a multiple linear regression of the logarithm of the hazard on the variables x_i, with the baseline hazard being an ‘intercept’ term that varies with time.

The quantities exp(b_i) are called hazard ratios (HR). A value of b_i greater than zero, or equivalently a hazard ratio greater than one, indicates that as the value of the ith covariate increases, the event hazard increases and thus the length of survival decreases.

Put another way, a hazard ratio above 1 indicates a covariate that is positively associated with the event probability, and thus negatively associated with the length of survival.

      Univariate Cox proportional hazards regression model

PROC PHREG DATA=dataset;
MODEL tte*cnsr(1)=treat /
TIES=EXACT RISKLIMITS ALPHA;RUN;

      Multivariate Cox proportional hazards regression model

PROC PHREG DATA=dataset;
MODEL tte*cnsr(1)=treat cov1 .. covk /
TIES=EXACT RISKLIMITS ALPHA;
RUN;

* tte represents variable containing event/censoring times;
* cnsr represents censoring variable (0=event, 1=censored);
* treat represents treatment group variable (0=placebo, 1=treatment);
* cov1 to covk represent covariates other than treatment group;

Stratified Multivariate Cox Regression (Stratified Adjusted Proportional Hazards Regression)

The stratified adjusted proportional hazards regression model and its assumptions are illustrated by:

h (.) hazard function as a function of time (relative to the start date), the patient’s

stratum and covariates, and the unknown vector of regression parameters

h_0k(t) unspecified baseline hazard function for patients with covariate vector (0, .., 0) for stratum k at time t

x_jk vector of Baseline covariates for patient j in stratum k, with xjk1 representing

treatment coded as 0 (placebo) or 1 (treatment)

β: unknown regression parameter vector on the log-scale (β1 represents the log hazard ratio for treatment).

The hazard ratio for the i-th covariate is a multiplicative constant exp(βi) comparing the hazard functions between the levels of the i-th covariate. Baseline hazard functions are allowed varying across strata without any restrictions. However, the hazard ratio comparing a covariate (including treatment) is assumed to be common across strata. The estimates of βi are adjusted for the other covariates in the model.

SAS PROC PHREG (with the TIES=EXACT option for the “exact” handling of ties) is

used to estimate the hazard ratios based on the partial maximum likelihood function; a Wald-test based two-sided CI is requested by the RISKLIMITS option (additional options controlling the output may be added):

PROC PHREG DATA=dataset;
MODEL tte*cnsr(1)=treat cov1 .. covk /
TIES=EXACT RISKLIMITS ALPHA;

STRATA strat1 .. stratj;

RUN;

* tte represents variable containing event/censoring times;
* cnsr represents censoring variable (0=event, 1=censored);
* treat represents treatment group variable (0=placebo, 1=treatment);
* cov1 to covk represent covariates other than treatment group;
* strat1 to stratj represent stratification variables;

Unstratified log-rank test and stratified log-rank test

The logrank test, or log-rank test, is a hypothesis test to compare the survival distributions of two samples. It is a nonparametric test and appropriate to use when the data are right skewed and censored (technically, the censoring must be non-informative). It is widely used in clinical trials to establish the efficacy of a new treatment in comparison with a control treatment when the measurement is the time to event (such as the time from initial treatment to a heart attack).

There are unstratified log-rank test and stratified log-rank test (as described below). In a clinical trial with stratified randomization, the stratified log-rank test is commonly used and the stratification factors used for randomization will be included in the log-rank test. 

Even for a study with time to event primary efficacy endpoint, the sample size calculation is usually based on the unstratifed log-rank test (i.e., the stratification factors are not considered). 

Log-rank test can provide a p-value for comparing the survival distributions of two samples (between two treatment groups), however, as described in an early post, "Splitting p-value and estimate of the treatment difference", the magnitude of the treatment difference (hazard ratio) needs to be calculated using Cox proportional regression - a semi-parametric method.

Unstratified log-rank test

Let S_T(t) and Sp(t) denote the survival functions for the treatment group and placebo group, respectively.

The null hypothesis

  H0: S_T(t) = Sp(t) for all t ≥ 0 “identical survival functions for both treatment groups”

is tested against the one-sided alternative hypothesis

  HA: ST(t) ≥ Sp(t) for all t ≥ 0 and S_T(t) > Sp(t) for at least some t > 0

        “survival function in the treatment group superior to the survival function in the placebo group”

The unstratified log-rank test can be conducted by SAS PROC LIFETEST where the STRATA statement includes only the treatment group variable (treat). The TIME statement includes a variable with times to event (TTE) and an indicator variable for right censoring (cnsr) with 1 representing censoring (additional options controlling the output may be added):

PROC LIFETEST DATA=dataset METHOD=KM;
TIME tte*cnsr(1);
STRATA treat;RUN;

As an output of the procedure, the rank statistic S and variance Var(S) are obtained. Under the null hypothesis, the test statistic Z = S/sqrt[Var(S)] is approximately normally distributed. The one-sided p-value is therefore obtained from normally distributed Z statistic and H0 is rejected if the p-value does not exceed the (nominal) significance level assigned to the test.

Stratified log-rank test

Let ST,k(t) and Sp,k(t) denote the survival functions for the treatment group and placebo group in stratum k, k=1,..,K, K = m1 x m2 x .. x mj where mi denotes the number of categories for stratification

factor i, 1≤ i ≤ j, respectively. The null hypothesis

  H0: ST,k(t) = Sp,k(t) for all t ≥ 0 and all k (“identical survival functions for both treatment groups in each stratum”) is tested against the one-sided alternative hypothesis

  HA: ST,k(t) ≥ Sp,k(t) for all t ≥ 0 and all k=1,..,K and ST,m(t) > Sp,m(t) for at least some t > 0 and some m (“survival function in the treatment group superior to the survival in the placebo group in at least one stratum”)

Log-rank tests are performed for each of the strata separately, obtaining the rank statistic Sk and variance Var(Sk) where k=1, 2, …, K. The stratified log-rank test statistic is constructed as Z = [S1 +…+ SK] / sqrt[Var(S1) +…+ Var(SK)]. Under the null hypothesis, Z is approximately normally distributed. The one-sided stratified log-rank p-value is therefore obtained from normally distributed Z statistic and H0 is rejected if p does not exceed the (nominal) significance level assigned to the test.

The stratified log-rank test can be conducted with SAS PROC LIFETEST where the STRATA

statement includes the j strata variables (strat_1, .., strat_j) and the GROUP option

includes the treatment variable (treat). The TIME statement includes a variable with times to event (TTE) and an indicator variable for right censoring (cnsr) with 1 representing censoring (additional options controlling the output may be added):

PROC LIFETEST DATA=dataset METHOD=KM;
TIME tte*cnsr(1);
STRATA strat_1 .. strat_j GROUP=treat;
RUN;

ICH E19 Optimisation of Safety Data Collection and FDA Guidance on Collecting Less Safety Data

The U.S. Food and Drug Administration is announcing the availability of a draft guidance entitled “E19 Optimisation of Safety Data Collection.” The guidance was prepared under the auspices of the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) and provides recommendations regarding appropriate use of a selective approach to safety data collection in some late-stage pre- or postmarketing studies of drugs where the safety profile, with respect to commonly occurring adverse events, is well understood and documented. The agency intends for the draft guidance to advance important clinical research questions through clinical investigations that collect relevant patient data. This will enable an adequate benefit-risk assessment of the drug for its intended use, while reducing the burden to patients from unnecessary tests that may yield limited additional information. Interested parties may submit comments to the docket until August 26, 2019.

FDA publishes ICH guidelines as FDA guidance. This guidance reflects just one element in the FDA’s work with regulatory authorities and industry associations from around the world to promote international harmonization of regulatory requirements under the ICH. One of the goals of harmonization is to identify and reduce differences in technical requirements for drug development among regulatory agencies. FDA is committed to seeking scientifically based harmonized technical procedures for the development and manufacture of pharmaceuticals.

• ICH (2019) E19 Optimisation of Safety Data Collection
• FDA Guidance for Industry (2016) "Determining the Extent of Safety Data Collection Needed in Late-Stage Premarket and Post approval Clinical Investigations"

Duke-Margolis Center for Health Policy organized a two-day workshop "Leveraging Randomized Clinical Trials to Generate Real-World Evidence for Regulatory Purposes"

Dr. Ellis Unger from FDA discussed the topic "Safety Monitoring in Clinical Trials for Generation of RWE" (see the video @1:10)

"Once a drug has been approved, its risks/harms are well-characterized; safety monitoring in studies for new indications can be less intensive (in may circumstances)"

"...about the defensive research, which is really a huge problem, which is that let's collect everything because some regulators might ask us in a year, ho, well, what were the CBCs? all these nitpicking questions that require mountains of data.

We put on a guidance in 2016. it's called, 'Determining the Extent of Safety Data Collection Needed in Late-Stage Premarket and Post approval Clinical Investigations'  What it basically talks about collecting less than usual safety data in those situations where the safety data are well characterized. My particular office oversees the Division of Cardiovascular and Renal products, and we talk with many sponsors. Look, don't collect all this stuff in your next trial. You want to do a cardiovascular outcome trial, that's great. But we already know how many headaches and hangnails a drug causes.  Just collect serious adverse events. and they thank us and they shake their heads and they walk out of the door. and then when they send in the final protocol, they are collecting everything. This happens time and time again and it dawned on us. well, maybe the problem is regulators across the pond and so about 4 years ago, I think, we proposed this topic to ICH collecting less than full safety data collection and it was adopted and I'm glad to say that the step 2 guideline is out and open for public comment. it is called "optimization of safety data collection".

And the interesting thing was that a lot of the resistance to this approach was not coming from the regulators across the pond or in Asian. It was coming from companies who were just firmly entrenched in the idea that they better collet it because they may need it. I would like someone who works for a company that has done one of these outcome studies for an anti-diabetic drug could tell me exactly what they learned by getting vital signs every, lab everything, every nonserious adverse event. Tell me what you learned and tell me what it costed you. I suspect that you didn't get much bang for your buck.

So this guideline is out there and I really do hope most of the people who have a vested interest take a look at it and there's a docket in the US and other countries where you can submit your comments Through official Pathways to get them considered."

Further Readings:

• FDA Seeks Comment on ICH Guideline on Optimizing Safety Data Collection
• FDA published ICH E19 for comments

Upversion of medical coding dictionaries (MedDRA and WHO-DD)

In clinical trials, adverse events, medical histories, concomitant medications (drug names) are usually entered as the free text fields (verbatim terms) in the case report forms. Data in free text format can not be systematically summarized and analyzed, medical coding becomes necessary to convert the free text fields into categories.

Adverse events and medical histories are coded using MedDRA (Medical Dictionary for Regulatory Activities). Drug names (generic or brand names) are coded using WHO-DD (World Health Organization – Drug Dictionary). Early dates, there are different dictionaries available for performing the medical coding. Nowadays, the coding dictionaries are fixed on MedDRA and WHO-DD.

Both MedDRA and WHO-DD dictionaries are updated periodically - specifically, MedDRA is updated twice a year and WHO-DD is updated four times a year.

When we do a clinical trial, we will start with the latest version of the coding dictionaries. The clinical studies usually take several years to complete. By the time we reach the completion of the study, we will do the database lock (i.e., no data changes after the database lock). The coding dictionaries selected for use at the beginning of the study will be several years old and become obsolete.

One way to resolve this issue is to perform the upversion (or up-version, upversioning) of the medical dictionaries. For a clinical trial (especially a trial with extended study duration), it is good to update the medical coding dictionaries periodically. If upversion can not be performed periodically during the study, it is better to do an upversion at least at the end of the study (before the database lock).

Historically, there was no requirement/mandate for upversion, different companies had different practice in terms of the upversion. Here is the survey result about the upversoin in practice.



However, upversion will soon become a mandate for both MedDRA and WHO-DD dictionaries.

There are two Federal Registers pertinent to the upversion of the MedDRA and WHO-DD (or WHODG) dictionaries: one for MedDRA and one for WHO-DD. Notice that WHO-DD is now called WHODG (World Health Organization Drug Global) in the Federal Register.

Electronic Study Data Submission; Data Standards; Support for Version Update of the Medical Dictionary for Regulatory Activities

“Generally, the studies included in a submission are conducted over many years and may have used different MedDRA versions to code adverse events. The expectation is that sponsors or applicants will use the most current version of MedDRA at the time of study start. However, there is no requirement to recode earlier studies. The transition date for support and requirement to use the most current version of MedDRA is March 15, 2018. Although the use of the most current version is supported as of this Federal Register notice and sponsors or applicants are encouraged to begin using it, the use of the most current version will only be required in submissions for studies that start after March 15, 2019….”

Electronic Study Data Submission; Data Standards; Support for Version Update of World Health Organization Drug Global

“FDA currently supports the use of WHODG for the coding of concomitant medications in studies submitted to CBER or CDER in NDAs, ANDAs, BLAs, and certain INDs in the electronic common technical document format. Generally, the studies included in a submission are conducted over many years and may have used different WHODG versions to code concomitant medications. The expectation is that sponsors and applicants will use the most current B3-format annual version of WHODG at the time of study start. However, there is no requirement to recode earlier studies. The transition date for support of the most current B3- format annual version of WHODG is March 15, 2018. Although the use of the current B3-format annual version of WHODG is supported as of this Federal Register notice and sponsors or applicants are encouraged to begin using it, the use of the most current B3- format annual version will only be required in submissions for studies that start after March 15, 2019.”

Additional Readings:

• Upversioning MedDRA Dictionary - Insights from a Seasoned Coder
• Challenges of differentMedDRA versions for integrated data and database
• B3/C3 formats upversioning

Retire Statistical Significance and p-value?

In the March issue of The American Statistician, there was a special issue with 43 papers about “Statistical Inference in the 21st Century: A World Beyond p < 0.05”. The discussion about using the p-values was picked up by the scientific communities and triggered a lot of discussions. Some of the articles were provocative: “Retire Statistical Significance”, “Abandon / Retire Statistical Significance”. American Statistician Association’s president, Karen Kafadar, has also discussed this issue in his ‘president’s corner’.

• P-values: To Own or Not to Own?
• Statistics and Unintended Consequences

For a long time, statisticians have been cautioned about the misuse of the p-values.

• Don’t become the slave of the p-values
• Don’t base your conclusions solely on whether an association or effect was found to be “statistically significant” (i.e., the pvalue passed some arbitrary threshold such as p < 0.05).
• Don’t believe that an association or effect exists just because it was statistically significant.
• Don’t believe that an association or effect is absent just because it was not statistically significant.
• Don’t believe that your p-value gives the probability that chance alone produced the observed association or effect or the probability that your test hypothesis is true.
• Don’t conclude anything about scientific or practical importance based on statistical significance (or lack thereof).

The intention of the special issue is to trigger a healthy debate about the p-values and statistical significance, trigger the development of better methods, and provide the educations about the appropriate use and interpretation of the p-values. However, there is a danger of the unintended consequences: non-statisticians may be confused about what to do. Worse, “by breaking free from the bonds of statistical significance” as the editors suggest and several authors urge, researchers may read the call to “abandon statistical significance” as “abandon statistical methods altogether”.

The drug development relies on the clinical trials to demonstrate the substantial evidence about the efficacy and the substantial evidence comes from adequate and well-controlled investigations.

“evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts, qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling or proposed labeling thereof”

For a common disease, two pivotal studies (with each showing a statistical significance at alpha = 0.05) have been the requirement for FDA (see FDA guidance "Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products")

FDA has applied more flexibilities in evaluating the evidence for drugs/biological products for treating rare diseases especially those with unmet medical needs. Frank Sasinowski has two articles discussing this issue.

• Quantum of Effectiveness Evidence in FDA’s Approval of OrphanDrugs
• Quantum of Effectiveness Evidence in FDA’s Approval of Orphan Drugs: Update, July 2010 to June 2014

If we need to avoid the overuse and misuse of p-values, we will need to start with the changes in the statute of the laws and changes in regulatory science.

In addition, the scientific journals and editors may judge the value of a paper based on the significance of the results and favors the studies with statistical significance for publication. However, this may be changed now. On July 18. 2019 issue of New England Journal of Medicine (NEJM), an editorial paper was published "New Guidelines for Statistical Reporting in the Journal".

"The new guidelines discuss many aspects of the reporting of studies in the Journal, including a requirement to replace P values with estimates of effects or association and 95% confidence intervals when neither the protocol nor the statistical analysis plan has specified methods used to adjust for multiplicity."

With NEJM leading the way, other journals may follow. We will see more reporting of the confidence intervals and less reporting of the p-values. 

Generate Real-World Data (RWD) and Real-World Evidence (RWE) for Regulatory Purposes

Real-world data (RWD) and real-world evidence (RWE) have been hot topics in the drug development field and in the statistical field. With the new technologies, electronic health records, and big data, it is no surprise that RWD and RWE are much discussed to be a new way to revolutionize the clinical trial design and the regulations.

FDA has its dedicated webpage for 'Real-World Evidence' and issued several guidelines for using real-world evidence to support the regulatory approvals. 

What is the Definition of RWD and RWE?

Real-world data (RWD) are the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. RWD can come from a number of sources, for example:

• Electronic health records (EHRs)
• Claims and billing activities
• Product and disease registries
• Patient-generated data including in home-use settings
• Data gathered from other sources that can inform on health status, such as mobile devices

Real-world evidence (RWE) is the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD. RWE can be generated by different study designs or analyses, including but not limited to, randomized trials, including large simple trials, pragmatic trials, and observational studies (prospective and/or retrospective).

Last week, Duke Margolis Center for Health Policy organized a symposium titled "Leveraging Randomized Clinical Trials to Generate Real-World Evidence for Regulatory Purposes". All presentation slides and the video recordings are available for free.

Day 1 focused on the study design using RWD and RWE for efficacy measure (presentation slides, presentation video)

Day 2 focused on safety monitoring using RWD and RWE (presentation slides, presentation video)

This is not the first Duke Margolis Center for Health Policy organizes the event for this topic. Here are some previous events.

Second Annual Duke-Margolis Conference on Real-World Data and Evidence

Enhancing the Application of Real-World Evidence In Regulatory Decision-Making DAY 1

Public Workshop: A Framework for Regulatory Use of Real-World Evidence

Additional Reading: 

• Real-World Evidence — What Is It and What Can It Tell Us? NEJM 2018
• FDA guidance (2018) "Use of Real-World Evidence to Support Regulatory Decision-Makingfor Medical Devices"

• FDA Discusses RWD, RWE With Industry, Academia
• Good practices for real‐world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR‐ISPE Special Task Force on real‐world evidence in health care decision making Pharmacoepidemiology and Drug Safety and Value in Health
• Real-World Evidence and Real-World Data for Evaluating Drug Safety and Effectiveness
• Gilead and Novartis Seek to Expand What FDA Considers as Real-World Data and Real-World Evidence