Derma: Silent Speech Interaction Using Transcutaneous Motion Sensing
Jun Rekimoto, Yu Nishimura
Abstract
Silent speech interaction (SSI) enables speech communication without uttering an actual voice and can have a potential to make speech interaction available in public places. However, commonly studied image-based lip-reading for SSI requires a camera in front of the face and thus it is not suitable for mobile use. Ultrasound imaging requires expensive and complex equipment. In contrast, we propose a much simpler method by skin motion sensing. Two small 6-DOF accelerometer/angular velocity sensors attached under the chin acquire 12-dimensional multidimensional information of skin motion caused by the silent utterance. With neural networks, 35 different silent commands are identified with a recognition rate of 94%. While previous lip-reading studies have normally inferred speech from images of video with vocal speech, this study also proposes a method of learning entirely from non-vocal speech only. Compared to previous studies, we consider the proposed solution is less visible, lightweight, and is not affected by lighting conditions.