EEG-Based Driver Fatigue Detection Using Spatio-Temporal Fusion Network With Brain Region Partitioning Strategy

Abstract

Detecting driver fatigue is critical for ensuring traffic safety. Electroencephalography (EEG) is the golden standard for brain activity measurement and is considered a good indicator of detecting driver fatigue. However, the current driver fatigue detection algorithm has limitations in mining and fusing the spatiotemporal characteristics of EEG signals. In this paper, we propose a multi-branch deep learning network named spatio-temporal fusion network with brain region partitioning strategy (STFN-BRPS) to improve the accuracy and robustness of driver fatigue recognition. Initially, we develop a recurrent multi-scale convolution module (RMSCM) comprising a multi-scale convolution sub-module, a CNN-Bi-LSTM sub-module, and a residual structure branch. RMSCM effectively extracts highly discriminative long short-term temporal feature information. Secondly, we propose a dynamic graph convolution module and a spatial graph edges’ importance weight assignment method based on brain region partitioning strategy, which can acquire intrinsic spatial feature information between electrodes. Thirdly, we design a feature fusion module (FFM) that utilizes channel attention to fuse long short-term temporal and spatial features. FFM learns and prioritizes the significance and relevance of each channel in the fused features. Finally, the fused spatio-temporal features are passed into the classification module to obtain the predicted driver fatigue state. Extensive comparison and ablation studies are conducted on EEG signals collected from real-world driving scenarios. The results demonstrate that the proposed STFN-BRPS model delivers superior classification performance compared to the mainstream methods. This study establishes a benchmark for EEG-based driver fatigue detection and related deep-learning modeling work.

Topics & Concepts