Multiple Endpoints in Clinical Trials (Final) - FDA Guidance for Industry

Today, the FDA finalized its guidance for industry "Multiple Endpoints in Clinical Trials". The draft version of this guidance was issued in 2017. The guidance is intended to help sponsors better understand FDA's current thinking about the issues related to the multiple endpoints and multiplicity issues for multiple endpoints, and different approaches in handling multiplicity issues. The guidance also discussed composite endpoints and multi-component endpoints. 

Typically, an adequate and well-controlled study will include only one primary efficacy endpoint and then multiple secondary efficacy endpoints, and additional exploratory endpoints. Exploratory endpoints are those endpoints for research purposes or for new hypotheses generation and not for the purpose of the product label. Primary and secondary efficacy endpoints can potentially be included on the product label. However, the testing hierarchy and sound approach for multiplicity adjustment must be pre-specified. The Fixed-Sequence Method in the appendix of this guidance seems to be commonly used. With Fixed-Sequence Method, the secondary efficacy endpoints will be tested only if the primary efficacy endpoint is statistically significant. The secondary efficacy endpoints are ranked or ordered based on the importance of the endpoints and the likelihood of getting statistically significant results. The next secondary efficacy endpoint will be tested only if the previous secondary endpoint is statistically significant. The testing hierarchy will stop once the hypothesis test for one of the secondary endpoints is not statistically significant.  

If the sponsor wants to include secondary endpoints in the product label, multiplicity adjustment for secondary endpoints must be included in the statistical analysis plan. 

In the section discussing the co-primary endpoints, "When Demonstration of Treatment Effects on Two or More Distinct Endpoints Is Recommended to Establish Clinical Benefit (Co-Primary Endpoints)", the examples of clinical trials with co-primary endpoints included in the draft version of this guidance were removed from the final guidance. For example, the draft guidance mentioned the following example and the final guidance did not:

Presumably, this is due to the revised FDA guidance "Early Alzheimer's Disease: Developing Drugs for Treatment" and the availability of the integrated scale - Clinical Dementia Rating Sum of Boxes (CDR-SB) Score:

"An integrated scale that adequately and meaningfully assesses both daily function and cognitive effects in early AD patients is acceptable as a single primary efficacy outcome measure. " 

"Common Statistical Methods for Addressing Multiple Endpoint-Related Multiplicity Problems" was included in the body of the guidance in the draft guidance and is now moved to the Appendix: Statistical Methods. The list of methods remains the same and includes the Bonferroni method; the Holm procedure; the Hochberg procedure; prospective alpha allocation scheme; the fixed-sequence method; resampling-based, multiple-testing procedures; gatekeeping testing strategies; and graphical approaches based on sequentially rejective tests. 

As a regulatory agency, FDA is conservative and tries to avoid false conclusions. Without adequate adjustment for multiplicity, the alpha level (type I error rate) can be inflated, and statistically, significant differences may be wrongly declared for an ineffective drug. In the summary of this guidance, FDA concludes: 

Previous Posts:

• Weighted Bonferroni Method (or partition of alpha) in Clinical Trials with Multiple Endpoints
• Co-primary endpoints and multiple primary endpoints

Risk difference and confidence interval for Analyses of AEs and Clinical Laboratory Data

Several years ago, I posted an article "Should hypothesis tests be performed and p-values be provided for safety variables in efficacy evaluation clinical trials?". There are some new development on this topic. 

Recently, FDA in collaboration with the Duke-Margolis Center for Health Policy hosted a one-day virtual meeting focused on advancing pre-market safety analytics. At this workshop, it was revealed that FDA Biomedical Informatics and Regulatory Review Science (BIRRS) Team was working on a document called "Standard Safety Tables and Figures: Integrated Guide". The document is currently posted on regulations.gov for public comments. The integrated guide proposed the mockup shells how the safety data analyses (adverse events and clinical laboratory data) should be displayed. Throughout all the proposed shells, we can see that a column for "'Risk Difference (%) (95% CI)" are included. Here are a couple of examples. 

If this integrated guide become official and is implemented, the future analyses for safety data (adverse events and clinical laboratory parameters) will be shifted from the pure summary statistics to summary statistics + point estimate and 95% confidence interval for risk differences. p-values and hypothesis testing should not be provided. 

Risk difference and its 95% confidence interval are provided for the descriptive purpose, not for inferential purpose. As stated in the integrated guide "These safety analyses are exploratory in nature and confidence intervals (CIs) for the risk difference presented here are not adjusted for multiplicity."

In AE tables, the sort order will be by the risk difference (from the highest to the lowest). In this way, the reviewers can easily identify the AEs with largest risk difference between two treatment groups. 

There are several ways in calculating the confidence interval for risk difference. The commonly used approach is Wilson score method - a method of estimating the population probability from a sample probability when the probability follows the binomial distribution.

There seems to be some differences between the regulatory requirement and the requirement by the medical journals. We continue to see the requests from journals like New England Journal of Medicine for providing the p-values for AE summary tables. In our published article, "Inhaled Treprostinil in Pulmonary Hypertension Due to Interstitial Lung Disease", we had to provide the p values for AEs and other safety endpoints for treatment group comparison per NEJM's editor's request. 

PDUFA, GDUFA, BSUFA, and MDUFA - FDA's User Fee Programs

For professionals who are working in drug development areas, the term PDUFA should be a very familiar term. As a matter of fact, FDA's action date or decision date to approve a new drug application (NDA) or biological license application (BLA) is called 'PDUFA date'. There are several trackers to track FDA's calendar for NDA/BLA approvals based on the PSUFA dates. 

PDUFA stands for 'Prescription Drug User Fee Amendments' and it is a program allowing FDA to collect the application fees from the sponsor. The PDUFA was created by Congress in 1992 and authorizes FDA to collect fees from companies that produce certain human drug and biological products. Since the passage of PDUFA, user fees have played an important role in expediting the drug approval process. In "Regulatory Education for Industry (REdI) Annual Conference 2022", Dr. Kevin Bugin from FDA presented "PDUFA Overview and Reauthorization".

"Timely review of the safety and effectiveness of the new drug applications and biologics license applications is essential to FDA mission to protect and promote public health. PDUFA is essential to these efforts. In fact, before PSDUFA is enacted in 1992, American's access to innovative new medicines lacked behind other countries. FDA’s premarket review process was unstaffed and unpredictable, and frankly slow. Agency lacks sufficient staff to perform a timely review and lacks in procedures and standards to ensure a rigorous, consistent, and predictable process.

So to tackle these challenges Congress passed PDUFA and this authorized FDA to collect industry user fees to hire additional staff and upgrade its information technology systems, processes, and so on, in return, it committed the agency to timelines for the application review process for new drugs without compromising its high standards for new drug safety efficacy and quality and, over the years as the PDUFA program has been reauthorized now six times going into its seventh time here hopefully there have been additional enhancements."

PDUFA must be reauthorized or renewed by congress every five years. Since its inception in 1992, it has been amended six times and we are now waiting for congress to authorize the 7th amendment (PDUFA VII). Notice that each PDUFA amendment may have its own name associated with it. 

PDUFA I - Original 

PDUFA II - FDAMA (Food and Drug Administration Modernization Act) 

PDUFA III - Public Health Security and Bioterrorism Preparedness and Response Act of 2002

PDUFA IV - FDAAA (Food and Drug Administration Amendments Act of 2007)

PDUFA V - FDASIA (Food and Drug Administration Safety and Innovation Act)

PDUFA VI - FDARA (FDA Reauthorization Act of 2017)

There are several sister programs designed with the same purpose as PDUFA, but for different approval pathways: GDUFA (Generic Drug User Fee Amendment), BSUFA (Biosimilar User Fee Amendment), and MDUFA (Medical Device User Fee Amendment). Recently, PDUFA VII was authorized by congress, and so were these sister programs. 

Along with the approval of PDUFA, GDUFA, BSUFA, and MDUFA, the user fee rates for various types of applications are updated and released by the FDA:

PDUFA: Prescription Drug User Fee Amendments

Prescription Drug User Fee Rates for Fiscal Year 2023

GDUFA: Generic Drug User Fee Amendment

Generic Drug User Fee Rates for Fiscal Year 2023

BSUFA: Biosimilar User Fee Amendment

Biosimilar User Fee Rates for Fiscal Year 2023

MDUFA: Medical Device User Fee Amendment

Medical Device User Fee Rates for Fiscal Year 2023

For companies that are well-funded and have steady revenues, these user fees are affordable. However, these user fees could be a burden for small companies. 

For drug development in diseases with orphan drug designation, the application fees for NDAs or BLAs are waived - this is an approach to encourage the companies to develop the drugs in orphan diseases. 

FDA's guidance "User Fee Waivers, Reductions,and Refunds for Drug andBiological Products" listed the situations where the user fees can be waived.