Psychological science today is locked in an arms race: a heated competition for superiority and status. This competition is not fought with weapons, material wealth, or even truth. It’s fought in publications.
Published papers have always served two purposes in the economy of scientific inquiry: They convey knowledge, but they’re also the currency that buys you status and a successful scientific career. We are hired, paid, and promoted largely on the basis of the papers we publish — not just their content but also their quantity (Lawrence, 2007). Of course, there are other metrics for success — grants, invited addresses and lectures, and so on — but publications are the primary currency of scientific distinction and standing.
Over the past several decades, the publication arms race has accelerated (Bornmann & Mutz, 2015). When I started my career almost 30 years ago, a few peer-reviewed publications could secure an academic job at a storied institution. Two or three peer-reviewed publications per year all but guaranteed tenure in the US and Canada (my colleagues tell me the situation was similar in Europe and Asia). Today, a tenure-worthy CV from 20 years ago might get you an assistant professorship (and at top institutions, it better include at least one publication in Science, Nature or maybe PNAS). A CV that used to get you a job now makes you competitive for a postdoctoral fellowship. Hell, some of my colleagues won’t even accept a student for graduate training without peer-reviewed publications in hand.
It’s long been known that incentive structures that favor quantity over quality, status over substance, are a risk to the progress and integrity of science (Edwards & Roy, 2017; Geman & Geman, 2016). Albert Einstein famously noted: “An academic career in which a person is forced to produce scientific writings in great amounts creates a danger of intellectual superficiality” (Isaacson, 2008, p. 79). Take a moment and consider your own publication record: How much of what you have published so far stands the test of time? How much truly chips away at psychology’s greatest mysteries and challenges?
Our incentive structure is also a risk to intellectual freedom (Barrett, 1998). We miss opportunities for discovery every time we’re pressured to conform to the prevailing wisdom rather than innovate and risk failure. Does the publication arms race create curious adventurers in the great unknown, or are we more like government contractors, racing to secure enough funding to support our laboratories?
These issues are deeply personal for me because my students, postdoctoral fellows, and younger colleagues face the same struggles and constraints. This is not the scientific culture I want to leave them with.
Before I make my next point, which focuses on the plight of young psychological scientists, I want it understood that I completely and unambiguously support efforts to tidy up our scientific practices. I am 100% in favor of having large, representative samples with sufficient power to test hypotheses. And I think published papers should include multiple studies (where possible) — preregistered of course — and those studies should replicate one another. These requirements are necessary for valid scientific practice. But these requirements, in the context of the publication arms race, further tighten the screws on our young scientists.
Consider the freshly minted assistant professor, building her first lab: To command the respect of her peers and eventually earn tenure, she must do more than resist the lure of p-hacking together a bunch of low-n-studies to produce valid scientific results. She is expected to publish five to 10 papers a year, each of which should contain several studies of large, samples from countries not considered Western, educated, industrialized, rich, and democratic (WEIRD). Imagine the number of experiments she’d have to run to achieve this outcome — not to mention the hours designing tasks, managing students and staff, analyzing data, and so on. Think about how hard it is to secure sufficient grant funding for even a couple of innovative studies, particularly for someone at the start of her career.
This catch-22 is enough to make young scientists leave the field before they even get started — which they do in increasing numbers (Gould, 2015; Anonymous Academic, 2018). It’s enough to make undergraduate students question whether they want to pursue a PhD in the first place — which they question with increasing frequency. To be honest, if I were a young scientist today, I would seriously consider investing my time and energy elsewhere.
The publication arms race is a result of cultural inheritance — scientists are trained with a set of norms, values, and practices that they pass on to their students, who then pass it on in turn (Smaldino & McElreath, 2016). A few bad apples may have deliberately published junk to get tenure, but most of the scientists who helped create the arms race did so unwittingly or unwillingly. What happens next is up to us. Sometimes we’re responsible for fixing things not because we’re to blame but because we’re the only ones who can.
How can we change the incentive structure in psychological science? It seems a task of Herculean proportions, with consequences that stretch beyond the internal workings of our science (i.e., how we hire, evaluate, promote, and fund ourselves) to the larger scientific landscape. We’ve faced onerous challenges before, most recently the replication crisis and its aftermath. Psychological science led the charge on that one, turning lemons into lemonade, creating what is now called the credibility revolution. And we can lead again. We are a hub science, after all, not just in content (Cacioppo, 2007), but also in process.
In the coming months, I’ll encourage the APS Board of Directors to take up these topics seriously. This means learning from those who study the process of science and the scientists themselves in the same way that we study the behavior of participants in our labs. It also means engaging with those who have creative ideas about how to reshape our incentive structure and mitigate the publication arms race. In the meantime, change can start with each individual.
Each of us, the next time we’re on a search or tenure-and- promotion committee, can commit to reading applicants papers instead of counting them. Each of us, when sitting down to write the next manuscript, or even better, to design the next set of experiments, can ask: Will this research contribute something of substance? Does it have a real possibility of moving psychological science forward or applying our science to help those in need? Each of us, when we encounter failure, can admit it freely, applauding our colleagues who do the same because being wrong sets the stage for important scientific discoveries (Firestein, 2012, 2015). And each of us, the next time we’re on a grant panel, can encourage research that has a high risk of failure but higher potential payoff: research of substantial creativity that seriously challenges the status quo. The future of psychological science depends on it.
Anonymous Academic (2018, February 16). Performance-driven culture is ruining scientific research. The Guardian. Retrieved from https://www.theguardian.com/higher-education-network/2018/feb/16/performance-driven-culture-is-ruining-scientific-research
Barrett, L. F. (1998). The future of emotion research. The Affect Scientist, 12, 6–8. Retrieved from https://www.affective-science.org/pubs/1998/future-of-emotion-research.pdf
Bornmann, L., & Mutz, R. (2015). Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. Journal of the Association for Information Science and Technology, 66, 2215–2222. https://doi.org/10.1002/asi.23329
Cacioppo, J. T. ( 2007). Psychology is a hub science. Observer, 20(8), 42. Retrieved from https://www.psychologicalscience.org/observer/psychology-is-a-hub-science
Edwards, M. A., & Roy, S. (2017). Academic research in the 21st century: Maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environmental Engineering Science, 34, 51–61. https://doi.org/10.1089/ees.2016.0223
Firestein, S. (2012). Ignorance: How it drives science. New York, NY: Oxford University Press.
Firestein, S. (2015). Failure: Why science is so successful. New York, NY: Oxford University Press.
Geman, D. & Geman, S. (2016). Science in the age of selfies. Proceedings of the National Academy of Sciences, 113, 9384–9387. https://doi.org/10.1073/pnas.1609793113
Gould, J. (2015). How to build a better PhD. Nature, 528, 22–25. https://doi.org/10.1038/528022a
Isaacson, W. (2008). Einstein: His life and universe. New York, NY: Simon and Schuster.
Lawrence, P. A. (2007). The mismeasurement of science. Current Biology, 17, R583–R585. https://doi.org/10.1016/j.cub.2007.06.014
Smaldino P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3, Article 160384. https://doi.org/10.1098/rsos.160384