An AIoT Framework With Multimodal Frequency Fusion for WiFi-Based Coarse and Fine Activity Recognition
Junxin Chen, Xu Xu, Tingting Wang, Gwanggil Jeon, David Camacho
Abstract
Benefiting from the progresses of sensing and sustainable computing technologies, recent years have witnessed the dramatic progresses of artificial intelligence of things (AIoT). As a typical AIoT application, WiFi-based human activity recognition has increasing popularities in smart homes. However, WiFibased action recognition often has unstable performance due to environmental interference. To this end, a robust deep learning framework called MSF-Net is proposed for coarse and fine activity recognition using channel state information (CSI) information. First, a dual-stream structure incorporating short-time Fourier transform and discrete wavelet transform is developed to highlight abnormal information in the CSI data. Then, a Transformer is employed as the backbone to effectively extract high-level features. In addition, an attention-based fusion branch is designed to enhance cross-model fusion. Experimental results show that MSF-Net achieves Cohens Kappa scores of 91.82%, 69.76%, 85.91%, and 75.66% on the SignFi, Widar3.0, UT-HAR, and NTU-HAR datasets, respectively. These performance records demonstrate the advantages of MSF-Net over existing methods for coarse and fine activity recognition based on WiFi data.