Why experimental variation in neuroimaging should be embraced
Gregory Kiar, Jeanette A. Mumford, Ting Xu, Joshua T Vogelstein, Tristan Glatard, Michael P. Milham
Abstract
In a perfect world, scientists would develop analyses that are guaranteed to reveal the ground truth of a research question. In reality, there are countless viable workflows that produce distinct, often conflicting, results. Although reproducibility places a necessary bound on the validity of results, it is not sufficient for claiming underlying validity, eventual utility, or generalizability. In this work we focus on how embracing variability in data analysis can improve the generalizability of results. We contextualize how design decisions in brain imaging can be made to capture variation, highlight examples, and discuss how variability capture may improve the quality of results. Brain imaging analysis lacks accessible ground-truth approaches, leading to varied results across the field. Embracing analytical variability may allow researchers to enhance the generalizability of findings and accelerate progress.