Tuesday, March 01, 2016

One-sided versus Two-sided test

For vast majority of clinical trials, two-sided tests are performed and two-sided p-values are presented. Once a while, we will see some study results presented with one-sided p-value. 

In a previous post "One-Sided Test in A Superiority Trial", an example from the RAPID study was given and the purpose of presenting the one-sided p-value seemed to be for looking better to the readers. Since then, the RAPID study results have been officially published in LANCET. However, in LANCET publication, the one-sided p-value was replaced with two-sided p-value. I can only guess that LANCET did not like the trick of presenting the one-sided p-value. Below is the comparison. Notice that for a one-sided p value of 0.017 (significance level is 0.025), the two sided p value is supposed to be 0.034 (significance level is 0.05).

The annual rate of lung density loss was significantly less in augmentation-treated patients (-1.45 +/- 0.24 units vs. -2.19 +/-0.25 units; p = 0.017, one-sided).

However, the annual rate of lung density loss at TLC alone was significantly less in patients in the A1PI group (–1·45 g/L per year [SE 0·23]) than in the placebo group (–2·19 g/L per year [0·25]; difference 0·74 g/L per year [95% CI 0·06–1·42], p=0·03)
In a recent paper by Sitbon et al "Selexipag for the Treatment of Pulmonary Arterial Hypertension", ones-sided p-value was presented. Here is what the paper says:
  •  Sample size calculation is based on at a one-sided type 1 error rate of 0.005.
  • Because of the alpha spending for one interim analysis, The final analysis used a one-sided significance level of 0.00499 instead of 0.005.
  • P values were calculated with the use of a one-sided log-rank test. 
Ironically, even though the one-side p-value was presented, when it came to the confidence interval, the two-sided confidence interval were presented.
FDA's statistical review has more details about how the statistical analyses were performed and the results were presented. In this study, one of the reasons for using the one-sided p-value could be the nature of the group sequential design. In group sequential design and the adaptive design, one-sided significant level is often used because it is easier for calculation.
In both cases, the statistical test was essentially the two-sided test even though the one-sided p-values were presented. The significance level was α/2 instead of α. I can only guess that the reason for presenting the one-sided p-value is to make the p-value look smaller (more impressive).


Gabriel said...

Dr Deng, what does it mean when you say "Because of the alpha spending for one interim analysis, The final analysis used a one-sided significance level of 0.00499 instead of 0.005."?

Is this done for multiplicity issues?

Web blog from Dr. Deng said...

Yes, they assigned a very small alpha spending for their interim analysis. Therefore, they still have 0.00499 instead of 0.005 alpha left for the final analysis.