Sonomyographic Prosthetic Interaction: Online Simultaneous and Proportional Control of Wrist and Hand Motions Using Semisupervised Learning
Xingchen Yang, Jipeng Yan, Zongtian Yin, Honghai Liu
Abstract
Human hands play a very important role in daily object manipulation. Current prosthetic hands are capable of mimicking most functions of the human hand, but how to interact with prosthetic hands based on human intentions remains an open problem. In this article, we proposed a wearable ultrasound-based interface to achieve simultaneous and proportional control <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> of wrist rotation (pronation/supination) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> and hand grasp (open/close). A semisupervised learning framework integrating principal component analysis and sparse Gaussian process regression (SPGP) was proposed to simplify the cumbersome model calibration, which is a key issue that hinders the practical application of existing simultaneous and proportional prosthetic control approaches. The proposed algorithms were verified with both offline and online experiments on 12 able-bodied subjects. The offline analysis showed that the first principal component of ultrasound features (PC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\#$</tex-math></inline-formula> 1) was inherently linear to wrist rotations and the SPGP was able to establish the mapping between ultrasound features and hand grasp kinematics with fewer training data. The online target achievement control test showed that the proposed method can achieve accurate control of a virtual prosthesis, with motion completion rate of 97.61 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 4.67%, motion completion time of 4.66 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 0.91 s, and stability error of 10.99 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.69 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> . This is the first study to achieve online simultaneous and proportional control of wrist and hand kinematics using ultrasound and semisupervised learning, paving the way for the era of muscle morphology-driven prosthetic control.