A Fast-Response Dynamic-Static Parallel Attention GCN Network for Body–Hand Gesture Recognition in HRI
Xiaofeng Guo, Qing Zhu, Yaonan Wang, Yang Mo, Jingtao Sun, Kai Zeng, Mingtao Feng
Abstract
Human–robot interaction (HRI) systems are crucial in robotics, natural, fast-response, and multimodal are the future trends in their development. However, current interaction methods have the following flaws: 1) slow response of action recognition algorithms for generic scenes, especially at the beginning stage; 2) insufficient feature extraction and fusion capabilities for spatiotemporal graph data; and 3) no good paradigm of body–hand recognition in HRI. To overcome these bottlenecks, we propose a fast-response graph convolutional network (GCN) for body–hand gesture recognition. First, we propose a dynamic-static parallel network for dynamic body gestures that is responsive and accurate. Second, we propose a spatiotemporal graph attention module to improve the graph data fusion effect in the dynamic-static network. Third, we implement a complete command module to form complete commands with body and hand information for interactions and control of the robot. Finally, extensive experiments on four datasets and real-world experiments were conducted to demonstrate that our network is capable of fast response and accurate recognition of dynamic body gestures at the beginning stage, verifying the effectiveness of skeleton-based body-hand gesture recognition, with a clear advantage over the state-of-the-art.