Recalibrating expectations about effect size: A multi-method survey of effect sizes in the ABCD study
Max M. Owens, Alexandra Potter, Courtland S. Hyatt, Matthew D. Albaugh, Wesley K. Thompson, Terry L. Jernigan, Dekang Yuan, Sage Hahn, Nicholas Allgaier, Hugh Garavan
Abstract
Effect sizes are commonly interpreted using heuristics established by Cohen (e.g., small: r = .1, medium r = .3, large r = .5), despite mounting evidence that these guidelines are mis-calibrated to the effects typically found in psychological research. Here we summarize Pearson’s correlations among all questionnaire and task data from the ABCD Study® to illustrate effect size distributions in a large, diverse sample of 9/10-year-old children. The median in-sample effect size was .03, and values at the first and third quartiles were .01 and .07. Effects were smaller for associations across instruments, content domains, and reporters, as well as when covarying for sociodemographic factors. To help modify researcher expectations, we provide benchmark examples for varying effect sizes. In summary, this report finds that empirically determined effect sizes from a notably large dataset are smaller than would be expected based on existing heuristics.