AFES: Attention-Based Feature Excitation and Sorting for Action Recognition
Mingliang Zhou, Jiahao Li, Xuekai Wei, Jun Luo, Huayan Pu, Wei Wang, Jingyuan He, Zhaowei Shang
Abstract
Action recognition in virtual reality (VR) is gaining importance as users seek more immersive interactions with virtual environments. Concepts such as augmented reality and the metaverse aim to bridge the gap between the real and virtual worlds, ensuring consistency and efficient feedback for actions in both realms. Previous works in action recognition have typically utilized spatial or temporal convolutions, but these methods often overlook key motion features that are critical for distinguishing between similar action types. In this paper, we propose attention-based feature excitation and sorting (AFES) blocks with hierarchically ordered residual connections to address spatiotemporal information in image sequences for multiscale action recognition tasks. First, we replace the single residual module in existing models with a multiscale learning module that improves feature utilization by adapting to the importance of each feature. Second, we introduce a novel action recognition framework by integrating our multiscale modules into mainstream models, enhancing their ability to recognize actions. Finally, we conduct extensive experiments on widely used datasets, which demonstrate that our multiscale action recognition algorithm outperforms state-of-the-art tracking algorithms.