Rodar: Robust Gesture Recognition Based on mmWave Radar Under Human Activity Interference
Can Jin, Xiangzhu Meng, Xuanheng Li, Jie Wang, Miao Pan, Yuguang Fang
Abstract
Using mmWave radar to conduct gesture recognition is a promising solution for human-computer interaction. Although many studies have shown initial success, two-fold problems still remain unsolved, namely, the high-strength human activity interference and the difficulty in handling similar gestures. In light of these, we develop a robust mmWave radar based gesture recognition system, Rodar, to achieve accurate recognition of similar gestures under high-strength human activity interference, where a Multi-view De-interference Transformer (MvDeFormer) network is proposed. Specifically, to deal with the strong human activity interference, we design a DeFormer module to capture the useful gesture features by learning different patterns between gestures and interference, thereby reducing the impact of interference. Then, we develop a hierarchical multi-view fusion module to first extract the enhanced features within each view, and effectively fuse them across various views for final recognition. To evaluate the proposed Rodar system, we construct a dataset with seven similar gestures under three common human activity interference scenarios. Experimental results show that the accuracy can achieve up to 93.01%. The code implementations are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Xlab2024/MvDeFormer</uri> .