Abstract striped circles dots colorful and shadow on white background.

Researchers Explore Life Beyond p < .05

Several APS Fellows are contributors to a special journal issue focused on the overuse of p values in statistical inference in science.

The 43 articles published in the latest issue of The American Statistician, the flagship journal of the American Statistical Association (ASA), distill a variety of opinions and insights on the use of p values in null hypothesis significance testing.

The collection of papers explores limitations of p values, alternatives to p < .05 as the “gold standard” for determining statistical significance, and recommendations for supplementing and replacing p values. All of the articles are open access.

Contributing authors include APS Fellows Ulf Böckenholt and Jennifer Tackett of Northwestern University, Joachim Krueger (Brown University), Geoff Cumming (La Trobe University, Australia), Jessica Utts (University of California, Irvine), Brian Haig (University of Canterbury, New Zealand), and Eric-Jan Wagenmakers (University of Amsterdam). The special issue follows ASA’s 2016 statement encouraging researchers to be wary of statistical claims based on p values alone.

APS Fellow Geoff Cumming and other psychological scientists have long been advocates for alternatives to null-hypothesis significance testing. These alternatives — dubbed the “new statistics” — include estimation based on effect sizes, confidence intervals, and meta-analysis. In an article in Psychological Science, Cumming outlines eight steps that researchers can follow to pursue research questions using a new-statistics approach.

Transitioning from p values to estimation and other techniques will produce a research literature that reflects a cumulative quantitative discipline, Cumming says. To help researchers adopt these practices, he developed a six-part video tutorial, available exclusively from APS.

Authors are also encouraged to use new-statistics practices when submitting to APS journals, including the flagship journal, Psychological Science. This position represents one of several initiatives aimed at advancing transparent and replicable research. In 2017, APS launched a new journal, Advances in Methods and Practices in Psychological Science, with the mission of bridging and integrating conversations on scientific best practices in various areas of psychological science. APS also offers an online methodology center, a one-stop shop for news and practical insights into the latest developments in research methods and practices.

Researchers seeking a hands-on learning experience also have the opportunity to learn new statistical techniques and methodological approaches through workshops at the APS Annual Convention. The workshop offerings at the 2019 Annual Convention in Washington, DC cover a variety of topics, including the new statistics and Bayesian statistics.

Learn more about the APS Annual Convention program, including workshops, invited talks, cross-cutting theme programs, and other special events at
https://www.psychologicalscience.org/conventions/annual

References

Cumming, G. (2013). The new statistics: Why and how. Psychological Science, 25,7-29. doi:10.1177/0956797613504966

Eich, E. (2013). Business not as usual. Psychological Science, 25, 3-6. doi:10.1177/0956797613512465

Wasserstein, R. L., Schirm, A. L., & Azar, N. A. (Eds.). (2019). Statistical inference in the 21st century: A world beyond p < 0.05 [Special issue]. The American Statistician, 72(71).

Comments

It’s great to see this shout-out by APS for the wonderful collection of “post p<.05" articles in the special issue of TAS. Also worth a mention is the Comment (and Editorial) in Nature released at the same time, to mark the TAS issue. See
https://www.nature.com/articles/d41586-019-00857-9

Altmetric tweeted that that Comment had hit an attention score of 12,000 within a couple of days, the highest ever. See
http://bit.ly/2DhyJDI

Papers with titles like "Ditching statistical significance"! Wonderful! I salute the battalion of leading scholars who, since around 1950, have published justified excoriations of NHST. They are, much belatedly, being vindicated.

Roll on Open Science, and better more thoughtful ways of doing our statistics, including the use of estimation and meta-analysis.

Let me mention our (actually Bob wrote it) paper in the TAS collection: "The new statistics for better science: Ask how much, how uncertain, and what else is known" by R. J. Calin-Jageman and G. Cumming. See:
https://www.tandfonline.com/doi/pdf/10.1080/00031305.2018.1518266?needAccess=true

All strength to APS in its long-standing support for better research methods,

Geoff

Leave a Comment

Your email address will not be published.
In the interest of transparency, we do not accept anonymous comments.
Required fields are marked*

This site uses Akismet to reduce spam. Learn how your comment data is processed.