Ensuring Transparency and Trust in Supervised-Machine-Learning Studies: A Checklist for Psychological Researchers

Abstract

Machine-learning (ML) algorithms are being rapidly incorporated into the work of psychologists given their capability and flexibility in analyzing large-scale, complex, or otherwise messy data sets. In this context and in the spirit of open science, ML research should be conducted in a transparent, understandable, and ethical manner. However, publications by psychology researchers and practitioners show a troubling lack of consistency in reporting ML information. Given that ML offers a wide range of analytical options, in this article, we address an important need by providing a comprehensive, open-science checklist that specifies the information researchers should disclose at each stage of a supervised-ML project—from data collection and preprocessing to model selection, evaluation, interpretation, and code sharing. We hope that psychological researchers will benefit from this checklist when reporting ML results and will adapt and extend this checklist further in the future.