Enabling Natural Human–Computer Interaction Through AI-Powered Nanocomposite IoT Throat Vibration Sensor
Jiao Suo, Xiaomeng Yang, Y.F.V. Leung, Jianfei Wang, Meng Chen, Yifan Liu, Hui Sun, Zuobin Wang, Xiao Qiao, Wen J. Li
Abstract
Throat microphones show potential as wearable IoT sensors for voice and larynx movement recognition. By picking up vocal fold vibrations directly from the human throat, these can detect speech in noisy or windy environments where traditional microphones fail. Recent studies have investigated soft throat microphones due to their conformable fit with human skin. However, previous work has focused primarily on speaker recognition rather than the speech recognition capabilities of these sensors. This paper presents a flexible sponge-structured throat microphone that can accurately detect the fundamental frequency (F0) and F0 contour of human speech. Comparison with commercial contact microphones and air microphones demonstrates the proposed IoT throat microphone’s ability to capture vocal fold vibrations. While high throat vibration frequencies are damped by biological tissue filtering, the sensor can still achieve 89.80% accuracy in classifying 15 English words and 97.84% for 15 Chinese Mandarin words using signals lowpass filtered at 500Hz. Beyond voice recognition, a non-verbal “speaking bandage” system was also built to map throat movements like swallowing, coughing and mouth opening to words in real-time. This novel soft sensor demonstrates promise as an effective wearable for advanced larynx movement and voice recognition via IoT technologies. Potential applications include augmentative communication, rehabilitation, and human-computer interaction – opening new directions for assistive technologies powered by the subtleties of human speech production.