Adaptive Learning Against Muscle Fatigue for A-Mode Ultrasound-Based Gesture Recognition

Abstract

It is evident that the state of the art in multi-sensory hand gesture recognition indicates the superior performance of A-mode ultrasound (AUS) modality over its counterparts. Nevertheless, the practical implementation of AUS-based gesture recognition in real-life scenarios is impeded by its susceptibility to muscle fatigue, thereby limiting its long-term applicability. Therefore, it is crucial to explore the role of muscle fatigue in stability of AUS based long-term human machine interface. Endowing the recognition model with the adaptability to dynamic muscle fatigue state could potentially offer a viable solution to this issue. In this work, we propose an adaptive A-mode ultrasound-oriented neural network (A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> USNet), which comprises two components: a convolutional neural network-based adaptive feature extractor, and a category template-based adaptive classifier. A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> USNet possesses the ability to update the feature representation and decision boundary in response to muscle fatigue, consequently mitigating performance deterioration. To validate the effectiveness of our proposed method, nine subjects were recruited to participate in the experiment, and they were asked to perform fifteen subtle gestures for sixteen trials, with AUS data collected. The experimental results demonstrate that the proposed method can yield a remarkable improvement of 26.79% in classification accuracy during the late stage of muscle fatigue, and the proposed method significantly outperforms conventional adaptive linear discriminant analysis and incremental support vector machine. This study paves the way for establishing AUS as a viable option for long-term gesture recognition.

Topics & Concepts