One-Sided Test in A Superiority Trial

In this year’s ATS meeting, the results for a study “Zemaira in Subjects With Emphysema Due to Alpha1-Proteinase Inhibitor (API) Deficiency” (study acronym RAPID trial) was presented. The study was designed to test the superiority of Zemaira to Placebo in CT density endpoints. “According to study findings, the annual rate of lung density loss was significantly less in A1PI-treated patients (-1.45 +/- 0.24 units vs. -2.19 +/-0.25 units; p = 0.017, one sided).”  The interesting thing is that the one-sided test was used for testing differences between two treatment groups and one-sided  p-value was presented.   

The appropriateness of two- or one-sided tests has been the subject of controversy for over half a century. However, in clinical trials for regulatory approval, two-sided test is preferred and is almost uniformly adopted by the industry. ICH Guidance E9 “ STATISTICAL PRINCIPLES FOR CLINICAL TRIALS” clearly stated the followings:

“It is important to clarify whether one- or two-sided tests of statistical significance will be used, and in particular to justify prospectively the use of one-sided tests. If hypothesis tests are not considered appropriate, then the alternative process for arriving at statistical conclusions should be given. The issue of one-sided or two-sided approaches to inference is controversial and a diversity of views can be found in the statistical literature. The approach of setting type I errors for one-sided tests at half the conventional type I error used in two-sided tests is preferable in regulatory settings. This promotes consistency with the two-sided confidence intervals that are generally appropriate for estimating the possible size of the difference between two treatments.”

In RAPID trial mentioned above, the one-sided p-value was calculated as 0.017. This p-value would need to be compared with half of the conventionally significance level of 0.05. While p value of 0.017 is still statistically significant comparing to 0.025, people may wonder while two-sided p-value was not calculated for comparison to 0.05. In superiority trial, a p-value from one-sided test needs to be compared with 0.025 and a p-value from two-sided test needs to be compared with 0.05. Presenting the one-sided p-value may give a false impression that the study result is more significant since only smaller p-value is presented and the smaller (half) significance level is not presented. If the significant level is clearly stated and is presented along with the p-value, there will be no difference in understanding and interpretation of the results no matter whether one-sided or two-sided p-values are presented. If the one-sided p-value is 0.029, the result will not be statistically significant since the one-sided p-value needs to be compared to 0.025 instead of 0.05 even though it may give an wrong impression of a statistical significance.

Typically, clinical trials are designed as using two-sided test for primary efficacy endpoint. Randomized, controlled clinical trials are conducted due to the uncertainty of experimental treatment better than the comparator – equipoise. The statistical test must also consider the possibility (or probability) of experimental treatment is better than comparator or comparator is better than experimental treatment. This justifies the use of two-sided test.

If the benefit of experimental treatment is known to be better than the comparator - lack of equipoise, the one-sided test could be used, but the clinical trial would not be ethical to be conducted due to the lack of equipoise.