How Does Model (Mis)Specification Affect Statistical Power, Type I Error Rate, and Parameter Bias in Moderated Mediation? A Registered Report

Abstract

Moderated mediation models are commonly used in psychological research and other fields to model when and how effects occur. Researchers must choose which paths in the mediation model are moderated but may struggle to decide whether to include too many moderated paths (a maximalist approach) or too few (a minimalist approach). In this registered report, we investigate the impact of model specification on statistical power, Type I error rate, and parameter bias for the index of moderated mediation. In a systematic review of moderated mediation articles published over 1 year, we found that six model specifications accounted for 85% of analyses and that the median sample size was 285. We then conducted a Monte Carlo simulation study to examine the effects of model specification on power and Type I error rate; results were analyzed using multilevel logistic regression. Relative to the data-generating process, analysis models could be correctly specified, overspecified, underspecified, or completely misspecified. Overspecified models often showed lower statistical power than correctly specified models but relatively low parameter bias. Underspecified models generally had lower power and often unacceptably high parameter bias. Completely misspecified models showed inflated Type I error rates in some cases and sometimes unacceptable parameter bias. Based on these results, we recommend that researchers tend toward maximalist approaches to reduce parameter bias while acknowledging the associated loss of power. Preregistration can help establish a priori model specification plans.