Dual attention based fusion network for MCI Conversion Prediction
Min Luo, Zhen He, Hui Cui, Phillip G. D. Ward, Yi-Ping Phoebe Chen
Abstract
Alzheimer’s disease (AD) severely impacts the lives of many patients and their families. Predicting the progression of the disease from the early stage of mild cognitive impairment (MCI) is of substantial value for treatment, medical research and clinical trials. In this paper, we propose a novel dual attention network to classify progressive MCI (pMCI) and stable MCI (sMCI) using both magnetic resonance imaging (MRI) and neurocognitive metadata. A 3D CNN ShuffleNet V2 model is used as the network backbone to extract MRI image features. Then, neurocognitive metadata is used to guide the spatial attention mechanism to steer the model to focus attention on the most discriminative regions of the brain. In contrast to traditional fusion methods, we propose a ViT based self attention fusion mechanism to fuse the neurocognitive metadata with the 3D CNN feature maps. The experimental results show that our proposed model achieves an accuracy, AUC, and sensitivity of 81.34%, 0.874, and 0.85 respectively using 5-fold cross validation evaluation. A comprehensive experimental study shows our proposed approach significantly outperforms all previous methods for MCI progression classification. In addition, an ablation study shows both fusion methods contribute to the high final performance. • A novel dual-attention fusion method for predicting mild cognitive impairment conversion. • Neurocognitive metadata used to guide spatial attention on discriminative regions of the brain. • ViT based self-attention fusion method to fuse neurocognitive metadata with the 3D CNN feature maps. • The proposed dual-attention based fusion network achieves new state-of-the-art performance.