How representative are neuroimaging samples? Large-scale evidence for trait anxiety differences between fMRI and behaviour-only research participants
Caroline J. Charpentier, Paul Faulkner, Eva Pool, Verena Ly, Marieke S. Tollenaar, Lisa Marieke Kluen, Aniek Fransen, Yumeya Yamamori, Níall Lally, Anahit Mkrtchian, Vincent Valton, Quentin J. M. Huys, Ioannis Sarigiannidis, Kelly Morrow, Valentina Krenz, Felix Kalbe, Anna Lena Cremer, Gundula Zerbes, Franziska Magdalena Kausche, Nadine Wanke, Alessio Giarrizzo, Erdem Pulcu, Susannah E. Murphy, Alexander Kaltenboeck, Michael Browning, Lynn K. Paul, Roshan Cools, Karin Roelofs, Luiz Pessoa, Catherine J. Harmer, Henry W. Chase, Christian Grillon, Lars Schwabe, Jonathan P. Roiser, Oliver J. Robinson, John P. O’Doherty
Abstract
Over the past three decades, functional magnetic resonance imaging (fMRI) has become crucial to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into fMRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, a behavioural effect discovered in a non-imaging context not replicating in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (N = 272), that fMRI participants differ from behaviour-only participants on one fundamental individual difference variable: trait anxiety. Analysing trait anxiety scores and possible confounding variables from healthy volunteers across multiple institutions (N = 3317), we found robust support for lower trait anxiety in fMRI study participants, consistent with a sampling or self-selection bias. The bias was larger in studies that relied on phone screening (compared with full in-person psychiatric screening), recruited at least partly from convenience samples (compared with community samples), and in pharmacology studies. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures or sampling strategies to mitigate this bias.