Call for Volunteers: Psychological Science REPEAT Network

Are you passionate about ensuring the reproducibility of scientific research? The journal Psychological Science is looking for volunteers to join REPEAT—our new network of computational reproducibility checkers.

Context

Psychological Science has introduced new policies to improve transparency and rigour (Hardwicke & Vazire, 2024), including a requirement that quantitative articles are computationally reproducible¹ (meaning the reported results can be independently recreated by repeating the original analyses on the original data). We are building a network of volunteers to help us verify the computational reproducibility of articles before they are published to help the authors to do this effectively.² Anyone who meets the requirements below is welcome to apply, including graduate students and early career researchers.

What will I do?

• Verify the computational reproducibility of articles³ submitted to Psychological Science.
• Provide constructive feedback to authors to enhance the reproducibility of their work.

How much work is involved?

• Articles will be assigned ad-hoc. You can reject assignments if you are too busy.
• Reproducibility checks often involve several rounds of back-and-forth with authors and the overall time can vary considerably; we estimate between 1 to 10 hours per manuscript.
• To join the REPEATnetwork, you should be willing to complete at least four reproducibility checks a year. Each year, we will refresh the network to see if you want to remain a member.

What are the requirements?

• Demonstrable experience writing reproducible analyses in psychology (or similar).
• High conscientiousness, attention to detail, and a commitment to scientific integrity.

How do I volunteer?

Please submit an expression of interest by completing this Google form. If you have any questions, please e-mail Tom Hardwicke ([email protected]), Senior STAR Editor at Psychological Science.

Notes

¹ This requirement has some exceptions; for example, when data cannot be shared for ethical reasons.

² Currently, these checks are handled by the journal’s Statistics, Transparency, and Rigour (STAR) team; however, we believe a larger network will help us to access a broader range of analysis and software capabilities and meet demand more effectively. Specifically, your goal will be to reproduce all numerical values reported in the manuscript using the analysis scripts and data shared by the authors. Authors are asked to provide well organised and clearly documented files that explain step-by-step how to reproduce the results reported in the manuscript. If you cannot easily reproduce the results, you will not have to engage in extensive detective work to understand why—you can let the authors know what the problem is and ask them to fix it. Most researchers have not been taught how to make analyses reproducible, so you may need.