Advancing cognitive assessment in telemedicine: Validity and reliability of the telephone 10‐point cognitive screener

Abstract

The Coronavirus Disease 2019 pandemic demanded strategies to optimize remote cognitive assessments.1-3 Previous work proposed improving older adults' access to cognitive evaluations through telephone or video interviews.2 However, some obstacles still frustrate the implementation of tools in real-life remote assessments.3 For example, the Telephone Montreal Cognitive Assessment, to be applicable, had to remove items requiring visual cues or pen and paper and takes about 10 min to administer.3, 4 The 10-point cognitive screener (10-CS), a validated 2-min cognitive assessment, combines temporal orientation, verbal fluency, and word recall tests.5 This tool does not include motor, writing, or drawing tasks, configuring a promising candidate for remote assessments.6, 7 Therefore, we investigated the validity and reliability of the Telephone 10-CS for detecting cognitive impairment and dementia, comparing its performance with in-person assessments. We completed a cross-sectional study in the Memory Clinic of the Geriatrics Division at the University of Sao Paulo Medical School, Brazil.5 We screened patients aged ≥60 years consecutively referred for suspected cognitive impairment between June 2019 and March 2020. We excluded candidates with moderate or severe dementia, delirium, psychotic disorders, and severe disabilities precluding the interviews. The local Research Ethics Committee approved the study, and participants or their proxies provided written consent. Two geriatricians experienced in cognition classified patients as having normal cognition, cognitive impairment no dementia (CIND), or dementia using information from a 90-min assessment, which included clinical evaluation and a neuropsychological battery that assessed memory, attention, language, visuospatial and executive functions (Table S1 describes the complete protocol). Additionally, a global cognition z-score was estimated by averaging the z-scores from all cognitive tests and then standardizing each participant's average using the sample's mean and standard deviation.8 The 10-CS, combining three-item temporal orientation, semantic verbal fluency, and three-word recall tests, classified patients as having normal cognition (≥8 points), possible (6–7 points), or probable cognitive impairment (≤5) after adjustment for lower literacy levels.5 Investigators completed the in-person 10-CS as part of the neuropsychological battery. A blinded investigator conducted the Telephone 10-CS 1 week after patients visited the clinic, following strict protocols (Figure S1). We calculated Spearman's rank correlation between the 10-CS versions and global cognition z-score to investigate construct validity. We assessed the intermodal reliability between 10-CS versions using intraclass correlation coefficients. We also examined the possibility of systematic bias by employing Bland–Altman analysis.9 Finally, we computed areas under the receiver operating characteristic curves (AUC) to estimate the accuracy of 10-CS versions in identifying cognitive impairment and dementia, reporting sensitivity, specificity, and likelihood ratios for each Telephone 10-CS score. We included 84 patients (mean age = 78.7 ± 6.6 years, female = 70%). Although the cognitive scores varied significantly according to cognitive status, we did not observe significant differences between groups in sociodemographic and clinical measures (Table 1). Similar to the in-person 10-CS, the correlation between the Telephone 10-CS and global cognition z-score was moderate (Spearman's coefficient = 0.59; 95% confidence interval = 0.43–0.71) (Figure S2). The intraclass correlation coefficient between the two 10-CS versions was 0.72 (95% confidence interval = 0.60–0.81), and the Bland–Altman analysis did not find systematic bias (Figure S3). The Telephone 10-CS had excellent accuracy in identifying cognitive impairment and dementia, a performance comparable to the in-person 10-CS (Figure 1). The best balance between sensitivity (83%) and specificity (80%) to detect dementia was the cutoff of ≤5 points in the Telephone 10-CS (Table S2). The Telephone 10-CS accurately identified cognitive impairment and dementia in older adults referred for suspected cognitive impairment. It also presented good reliability with in-person 10-CS. Unlike several cognitive tests, the 10-CS is quick, free of charge,2-4 does not rely on expensive technological resources or additional supplies (pen, paper, image sheets), and is not influenced by a patient's ability to read, write, or draw. Such characteristics make it a valuable tool when remote assessments are necessary (e.g., limited transportation in rural and less developed areas and adequate follow-up of homebound patients), improving older adults' access to cognitive evaluations.10 Our study had limitations. It was a single-center study with a sample limited to patients having suspected cognitive impairment, which does not reflect the characteristics of a broader population. There were also limitations to applying phone-based assessments—excluding patients with severe disabilities (e.g., hearing loss) and the inability to ensure the patient was not aided while completing tests.3, 4 However, we consider the short time to complete 10-CS hinders such aids. In conclusion, our results supported the validity and reliability of the Telephone 10-CS as an effective tool for remote cognitive assessment. Providers should be open to new alternatives like the Telephone 10-CS that could improve healthcare access and equity. Rico, Apolinario was involved in study concept and design, data acquisition, data analysis, data interpretation, and manuscript preparation. Aliberti was involved in data analysis, data interpretation, and manuscript preparation. Magaldi and Busse were involved in acquisition of data, data interpretation, and manuscript preparation. Silva, Suemoto, and Avelino-Silva were involved in data interpretation and manuscript preparation. Márlon Juliano Romero Aliberti is supported by a scholarship from HCFMUSP with funds donated by Nubank under the #HCCOMVIDA scheme. The funder had no role in the design, methods, subject recruitment, data collection, analysis, preparation, review, or approval of the manuscript and decision on its submission. The authors declare that there is no conflict of interest. The funder had no role in the design, methods, subject recruitment, data collection, analysis, preparation, review, or approval of the manuscript and decision on its submission. Table S1. The 90-min assessment including clinical evaluation and comprehensive neuropsychological battery for defining the cognitive status. Table S2. Sensitivity, specificity, and likelihood ratios for the education-adjusted scores of the 10-point Cognitive Screener according to each cutoff point. Figure S1. Instructions for administration and scoring of the 10-point Cognitive Screener (10-CS). Figure S2. Scatter plots illustrating the correlation between the in-person and Telephone 10-CS scores and the global cognition z-scores. Figure S3. Scatterplot and Bland–Altman plot for intermodal agreement between the Telephone 10-CS and in-person 10-CS. Additional references. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

Topics & Concepts