Sunday, August 29, 2021

Retire Statistical Significance and p-value? - Revisited

In 2019, there was a public debate about the use or misuse of statistical significance and p-values. I wrote a post about it "Retire Statistical Significance and p-value?"

Obviously, the hypothesis testing, statistical significance, and p-value are still the cornerstone of our clinical trials, the basis for judging if a clinical trial is a success, and the basis for the decision-making for regulatory approvals by the FDA and other regulatory authorities. 

In the latest issue of AMSTATNews and also the Annals of Applied Statistics, there is an article "The ASA President’s Task Force Statement on Statistical Significance and Replicability". The statements confirmed that the significance and p-values are here to stay. 
"P-values are valid statistical measures that provide convenient conventions for communicating the uncertainty inherent in quantitative results. Indeed, p-values and significance tests are among the most studied and best understood statistical procedures in the statistical literature. They are important tools that have advanced science through their proper application."

"p-values and significance tests, when properly applied and interpreted, increase the rigor of the conclusions drawn from data. Analyzing data and summarizing results are often more complex than is sometimes popularly conveyed. Although all scientific methods have limitations, the proper application of statistical methods is essential for interpreting the results of data analyses and enhancing the applicability of scientific results."

I would say the following about the significant tests and p-values:

  • We embrace it, not abandon it. 
  • We focus on the appropriate use and interpretation
  • We should not become the slave of the p-values

A quote from  Alfred Marshall, 1885:

"The most reckless and treacherous of all theorists is he who professes to let facts and figures speak for themselves, who keeps in the background the part he has played, perhaps unconsciously, in selecting and grouping them."

No comments: