Machine learning‐assisted screening for cognitive impairment in the emergency department
Simon R. Yadgir, Collin J. Engstrom, Gwen Costa Jacobsohn, Rebecca K. Green, Courtney M.C. Jones, Jeremy T. Cushman, Thomas V. Caprio, Amy Kind, Michael Lohmeier, Manish N. Shah, Brian W. Patterson
Abstract
BACKGROUND/OBJECTIVES: Despite a high prevalence and association with poor outcomes, screening to identify cognitive impairment (CI) in the emergency department (ED) is uncommon. Identification of high-risk subsets of older adults is a critical challenge to expanding screening programs. We developed and evaluated an automated screening tool to identify a subset of patients at high risk for CI. METHODS: In this secondary analysis of existing data collected for a randomized control trial, we developed machine-learning models to identify patients at higher risk of CI using only variables available in electronic health record (EHR). We used records from 1736 community-dwelling adults age > 59 being discharged from three EDs. Potential CI was determined based on the Blessed Orientation Memory Concentration (BOMC) test, administered in the ED. A nested cross-validation framework was used to evaluate machine-learning algorithms, comparing area under the receiver-operator curve (AUC) as the primary metric of performance. RESULTS: Based on BOMC scores, 121 of 1736 (7%) participants screened positive for potential CI at the time of their ED visit. The best performing algorithm, an XGBoost model, predicted BOMC positivity with an AUC of 0.72. With a classification threshold of 0.4, this model had a sensitivity of 0.73, a specificity of 0.64, a negative predictive value of 0.97, and a positive predictive value of 0.13. In a hypothetical ED with 200 older adult visits per week, the use of this model would lead to a decrease in the in-person screening burden from 200 to 77 individuals in order to detect 10 of 14 patients who would fail a BOMC. CONCLUSION: This study demonstrates that an algorithm based on EHR data can define a subset of patients at higher risk for CI. Incorporating such an algorithm into a screening workflow could allow screening efforts and resources to be focused where they have the most impact.