Do Great Results in the Lab Hold Up in the Field?

This is a photo of a person in a field holding a pair of binoculars.It was good news in 1999, when Craig A. Anderson and his colleagues compared laboratory and field research on 38 topics in 21 meta-analyses and found a lot of agreement between the results. Greg Mitchell, a social psychologist at the University of Virginia School of Law, wanted to know if these findings would hold up in a bigger sample. In a paper published in Perspectives on Psychological Science, Mitchell replicated the Anderson study with 217 lab-field comparisons from 82 meta-analyses, in such areas as industrial-organizational (I-O), social, consumer, and developmental psychology.

The results: “On one level, there is good news: a high degree of correspondence between findings observed in the lab and those found in the field,” Mitchell says.  “But if you look more closely, there are major variations. I-O led the pack by a long shot, social psychology did worse, and most other sub-disciplines fell somewhere in between.”

If you extract the I-O stats from the batch, the overall correlation between lab and field results drops considerably.  And in 30 of the 217 comparisons, the results in the field were the opposite of those in the lab.  Of these reversals, the majority came from social psychology. Laboratory studies of gender differences fared particularly poorly when results were tested under more realistic conditions.

The lesson? “We need to be conducting more field studies,” says Mitchell, and scientists should attempt to recreate as much of the field as possible in the lab, he says. “Because there’s a nontrivial chance the lab will point us in the wrong direction.”

APS regularly opens certain online articles for discussion on our website. Effective February 2021, you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Any opinions, findings, conclusions, or recommendations present in article comments are those of the writers and do not necessarily reflect the views of APS or the article’s author. For more information, please see our Community Guidelines.

Please login with your APS account to comment.