Natural language processing models reveal neural dynamics of human conversation
Jing Cai, Alex E. Hadjinicolaou, Angelique C. Paulk, Daniel J. Soper, Tian Xia, Alexander F Wang, John D. Rolston, R. Mark Richardson, Ziv M. Williams, Sydney S. Cash
Abstract
Through conversation, humans engage in a complex process of alternating speech production and comprehension to communicate. The neural mechanisms that underlie these complementary processes through which information is precisely conveyed by language, however, remain poorly understood. Here, we used pre-trained deep learning natural language processing models in combination with intracranial neuronal recordings to discover neural signals that reliably reflected speech production, comprehension, and their transitions during natural conversation between individuals. Our findings indicate that the neural activities that reflected speech production and comprehension were broadly distributed throughout frontotemporal areas across multiple frequency bands. We also find that these activities were specific to the words and sentences being conveyed and that they were dependent on the word’s specific context and order. Finally, we demonstrate that these neural patterns partially overlapped during language production and comprehension and that listener-speaker transitions were associated with specific, time-aligned changes in neural activity. Collectively, our findings reveal a dynamical organization of neural activities that subserve language production and comprehension during natural conversation and harness the use of deep learning models in understanding the neural mechanisms underlying human language. How the brain supports speaking and listening during conversation of its natural form remains poorly understood. Here, by combining intracranial EEG recordings with Natural Language Processing, the authors show broadly distributed frontotemporal neural signals that encode context-dependent linguistic information during both speaking and listening..