Real-time decoding of full-spectrum Chinese using brain-computer interface
Youkun Qian, C. Liu, P.Y. Yu, Xingchen Ran, Shangcheng Li, Qinrong Yang, Yan Liu, Lei Xia, Yijie Wang, Jianxuan Qi, Eyou Zhou, Junfeng Lu, Yuanning Li, Tiger H. Tao, Zhitao Zhou, Jinsong Wu
Abstract
Speech brain-computer interfaces (BCIs) offer a promising means to provide functional communication capacity for patients with anarthria caused by neurological conditions such as amyotrophic lateral sclerosis (ALS) or brainstem stroke. Current speech decoding research has predominantly focused on English using phoneme-driven architectures, whereas real-time decoding of tonal monosyllabic languages such as Mandarin Chinese remains a major challenge. This study demonstrates a real-time Mandarin speech BCI that decodes monosyllabic units directly from neural signals. Using the 256-channel microelectrocorticographic BCI, we achieved robust decoding of a comprehensive set of 394 distinct syllables based purely on neural signals, yielding median syllable identification accuracy of 71.2% in a single-character reading task. Leveraging this high-performing syllable decoder, we further demonstrated real-time sentence decoding. Our findings demonstrate the efficacy of a tonally integrated, direct syllable neural decoding approach for Mandarin Chinese, paving the way for full-coverage systems in tonal monosyllabic languages.