ADED: Method and Device for Automatically Detecting Early Depression Using Multimodal Physiological Signals Evoked and Perceived via Various Emotional Scenes in Virtual Reality
Mi Li, Yanbo Chen, Zeying Lu, Fan Ding, Bin Hu
Abstract
Severe depression often exhibits suicidal tendencies, making early identification and intervention crucial to prevent its further progression. This study focuses on developing a high-performance method and device for early detection of depression. We propose a hybrid framework for early depression detection that integrates multiple deep learning techniques and ensemble learning. This framework features a dual Bidirectional Temporal Convolutional Network (BiTCN) to encode both local and global causal relationships, a Bidirectional Long Short-Term Memory network (BiLSTM) to capture long-term dependencies and contextual relationships, an emotional cross-attention module to encode the significance of different emotions, a multimodal feature cross-attention mechanism to prioritize various feature modalities, and an ensemble learning method to decode and infer depression detection. The input signals include pupil waves and pulse rate variability (PRV) signals, measured during both calm (non-emotional) and emotional states such as sadness, happiness, fear, and tension. To enhance the generalization capability of the model, data augmentation techniques are applied to the training dataset. Test results show that detection performance based on emotional cues is superior to that based on non-emotional cues (calm). Notably, the fusion of pupil waves and PRV signals with emotional cues has achieved state-of-the-art performance in depression detection. These findings highlight the crucial role of emotional signals in improving the performance of depression detection. The end-to-end high-performance Automatic Detection of Early Depression Device (ADED) developed in this study can serve as an early detection tool, thereby promoting the potential application of artificial intelligence technology in mental health screening and clinical practice.