Control for Type I Error (or Adjustment for Multiplicity) for Secondary Endpoints

In clinical trial protocols, we usually specify one or more primary efficacy endpoints, then a list of secondary efficacy endpoints, and then more tertiary endpoints or exploratory endpoints. It is pretty standard that the primary efficacy endpoints are those the hypothesis testing or inferential statistics and sample size estimation are based on. If we have more than one primary efficacy endpoints, we will need to adjust for multiple tests or multiplicity. It is also clear that the tertiary or exploratory endpoints are for hypothesis generating or more bluntly for fishing expedition.  

Questions we are asked very often are:

• For secondary efficacy endpoints, do we need to do formal hypothesis testing?
• If so, do we need to adjust for multiple tests and multiplicity?
• What is the purpose of controlling the type I error for secondary efficacy endpoints?  

While controlling Family Wise Error Rate (FWER) for primary efficacy endpoints (if multiple test situation exists) is necessary, controlling FWER for secondary efficacy endpoints is often questioned.

Controlling the FWER for secondary efficacy endpoints is valuable or is necessary if we are planning to include the secondary efficacy endpoints in the product label if the product is approved. In other words, if we would like to make the claim of the benefits based on the secondary efficacy endpoints. It is advisable to perform the formal hypothesis testing with controlling for FWER.

In FDA’s guidance for industry “Clinical Studies Section of Labeling for Human Prescription Drug and Biological Products — Content and Format”, there is a statement about the primary and secondary endpoints:

§   Primary and Secondary Endpoints: The terms primary endpoint and secondary endpoint are used so variably that they are rarely helpful. The appropriate inquiry is whether there is a well-documented, statistically and clinically meaningful effect on a prospectively defined endpoint, not whether the endpoint was identified as primary or secondary.

FDA does not care whether or not a study endpoint is called primary or secondary. However, if the information from these endpoints are used for supporting evidence and for product label, the endpoints need to be predefined and tests for these endpoints need to be controlled for the overall alpha or overall type I error.

Even though FDA does not care about the terminology of primary/secondary endpoints, it is still very common in practice that the clinical trials (especially the industry-sponsored clinical trials) specify primary, secondary, and exploratory endpoints.

In a presentation by Kathleen Fritsch from FDA “Multiplicity Issues in FDA-Reviewed Clinical Trials”, the difference between primary, secondary, and exploratory efficacy endpoints are clearly specified and it is stated that the secondary efficacy endpoints may be included in the product label if multiplicity issue is addressed.

It is not needed or rarely needed to control for FWER for exploratory efficacy endpoints.

In EMA’s guidance “Points to Consider on Multiplicity Issues in Clinical Trials”, adjustment for multiplicity is explicitly required if the secondary variables are used for additional claims.

A slide presentation regarding EMA guidance further explained the secondary efficacy endpoints for claims.

            Norbert Benda (2013) The update of the multiplicityguideline

In FDA’s guidance on “Clinical Investigations of Devices Indicated for the Treatment of Urinary Incontinence”, the secondary endpoints were called out:

                Secondary Endpoints

      FDA believes secondary endpoint measures, by themselves, are not sufficient to characterize fully the treatment benefit. However, these measures may provide additional characterization of the treatment effect. Specifically, secondary endpoints can:

• supply background and understanding of the primary endpoints, in terms of overall direction and strength of the treatment effect;
• be the individual components of a composite primary endpoint, if used;
• include variables for which the study is underpowered to definitively assess;
• aid in the understanding of the treatment’s mechanism of action;
• be associated with relevant sub-hypotheses (separate from the major objective of the treatment); or
• be used to perform exploratory analyses.

       Assuming that the primary safety and effectiveness endpoints of the study are successfully met, we recommend you analyze the secondary endpoints to provide supportive evidence concerning the safety and effectiveness of the device, as well as to support descriptions of device performance in the labeling. To minimize bias, your protocol should prospectively identify all secondary endpoints, indicating how the data will be analyzed and what success criteria will be applied.

 Secondary Endpoint Analyses

We recommend your protocol prospectively define the statistical plan for performing secondary endpoint analyses in the event that the primary endpoint analysis has been successfully met. If the secondary endpoint analyses are intended purely as exploratory analyses, or are not intended to support the indication for use or device performance, we recommend you submit only simple descriptions of the analyses. If, on the other hand, any of the secondary endpoint analyses are intended to support the indication for use or the performance of your device in the labeling (e.g., comparing treatment and control groups using p-values or confidence intervals), we recommend you pre-specify this intention in your study protocol and describe in detail the statistical methods you plan to follow.

In summary, if we don’t perform the hypothesis testing for these secondary endpoints or not having appropriate control for overall type I error, we will lose the chance to claim the benefit on the secondary endpoints and to list secondary endpoints into the product label. Therefore, it is always a wise decision to perform the hypothesis testing for secondary endpoints with appropriate control for type I error.