Checking the Checkers: Evaluating the Validity of Attention Checks and Comparing Their Utility

We examined whether attention checks used in online studies truly measure participant attentional capacity. We demonstrate that all attention check types show a strong relationship with specific attentional measures, but that the most common type of check used by researchers exhibits shockingly low discriminability between high and low attention participants.