Rep-LOM: Multibranch Linear Operation Module With DFC Attention for Sensor-Based Human Activity Recognition

Abstract

Lightweight convolutional neural networks (CNNs) are well suited for human activity recognition (HAR) applications deployed on edge devices like smartphones and wearable devices. However, the small-kernel convolutional operation is able to only capture local details within a window region, thereby preventing further performance improvement. Though transformer-style models have recently employed self-attention to capture long-range dependence well, they require quadratic complexity w.r.t. the feature size, which is not computationally friendly for resource-constrained HAR deployment, largely encumbering actual activity speed. To address this challenge, this paper proposes a re-parameterized linear operation module (Rep-LOM), which integrates 1×1 convolutional kernels with linear transformations. The module utilizes a multi-branch training architecture, optimized in synergy with DFC attention mechanisms, to improve inter-channel feature interaction and long-range spatial dependency modeling. Though such 1×1 convolutional kernel with linear transformation is computationally efficient, directly paralleling the DFC attention with it would incur extra computational cost. Thus, we reduce the original feature size by subsampling it both horizontally and vertically, so that all the operations in DFC attention may be performed over the smaller features. During the inference phase, the training-time multi-branch topology can be converted back to a single-path deployment using structural re-parameterization techniques, thereby reducing computational cost and memory footprint, facilitating faster inference. Extensive evaluations across four public HAR benchmarks (UCI-HAR, USC-HAD, PAMAP2, and OPPORTUNITY) validate the effectiveness of our proposed method in balancing computational efficiency and recognition accuracy. Real-world HAR deployment is provided on a resource-constrained embedded platform. The relevant source code and demo video are released.

Topics & Concepts