APS Award Address

How to Detect and Correct the Lies That Data Tell

Frank L. Schmidt University of Iowa

In psychology and related fields there is an excessive faith in data as the direct source of scientific facts and an inadequate appreciation of how misleading most data are when accepted at face value. Because of distortions created by research artifacts such as sampling error, measurement error, dichotomization of measures, and other artifacts, observed data often lie to researchers. Detecting and correcting these lies requires use of meta-analysis methods that remove the biases and distortions created by these artifacts. Examples will be presented showing how this process often leads to conclusions radically different from those produced by naïve interpretations of research literatures based on statistical significance tests and discuss the implications for the attainment of cumulative scientific knowledge in psychology.