Members in the Media
From: Nature

Replication failures in psychology not due to differences in study populations

A large-scale effort to replicate results in psychology research has rebuffed claims that failures to reproduce social-science findings might be down to differences in study populations.

The drive recruited labs around the world to try to replicate the results of 28 classic and contemporary psychology experiments. Only half were reproduced successfully using a strict threshold for significance that was set at P < 0.0001 (the P value is a common test for judging the strength of scientific evidence).

The initiative sampled populations from across six continents, and the team behind the effort says that its overall findings suggest that the culture or setting of the group of participants is not an important factor in whether results can be replicated.

The reproducibility of research results — and psychology particularlyhas come under scrutiny in recent years. Several efforts have tried to repeat published findings in a variety of fields, with mixed outcomes.

The latest effort, called Many Labs 2, was led by psychologist Brian Nosek of the Center for Open Science in Charlottesville, Virginia. Nosek and his colleagues designed their project to address major criticisms of previous replication efforts — including questions about sampling and the assertion that research protocols might not be carried out properly in reproducibility attempts.

Researchers obtained the original materials used in each experiment, and asked experts — in many cases, the original authors of the studies — to review their experimental protocols in advance. Sixty different labs in 36 countries and territories then redid each experiment, providing combined sample sizes that were, on average, 62 times larger than the original ones. The results of the effort are posted today as a preprint1 and are scheduled to be published in Advances in Methods and Practices in Psychological Science.

Read the whole story (subscription may be required): Nature

More of our Members in the Media >


APS regularly opens certain online articles for discussion on our website. Effective February 2021, you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Any opinions, findings, conclusions, or recommendations present in article comments are those of the writers and do not necessarily reflect the views of APS or the article’s author. For more information, please see our Community Guidelines.

Please login with your APS account to comment.