Development of embedded performance validity indicators in the NIH Toolbox Cognitive Battery.
Christopher A. Abeare, László A. Erdődi, Isabelle Messa, Douglas P. Terry, William J. Panenka, Grant L. Iverson, Noah D. Silverberg
Abstract
To assess noncredible performance on the NIH Toolbox Cognitive Battery (NIHTB-CB), we developed embedded validity indicators (EVIs). Data were collected from 98 adults (54.1% female) as part of a prospective multicenter cross-sectional study at 4 mild traumatic brain injury (mTBI) specialty clinics. Traditional EVIs and novel item-based EVIs were developed for the NIHTB-CB using the Medical Symptom Validity Test (MSVT) as criterion. The signal detection profile of individual EVIs varied greatly. Multivariate models had superior classification accuracy. Failing ≥4 traditional EVIs at the liberal cutoff or ≥3 at the conservative cutoff produced a good combination of sensitivity (.57 to .61) and specificity (.92 to .94) to MSVT. Combining the traditional and item-based EVIs improved sensitivity (.65 to .70) at comparable specificity (.91 to .95). In conclusion, newly developed EVIs within the NIHTB-CB effectively discriminated between patients who passed versus failed the MSVT. Aggregating EVIs within the same category into validity composites improved signal detection over univariate cutoffs. Item-based EVIs improved classification accuracy over that of traditional EVIs. However, the marginal gains hardly justify the burden of extra calculations. The newly introduced EVIs require cross-validation before wide-spread research or clinical application. (PsycInfo Database Record (c) 2021 APA, all rights reserved).