Emotion-Aware Voice Assistants: Design, Implementation, and Preliminary Insights
Yong Ma, Yuchong Zhang, Miroslav Bachinski, Morten Fjeld
Abstract
The increasing prevalence of voice assistants in daily life has transformed human-technology interaction. However, current voice assistants fall short in their ability to comprehend users’ emotional states through speech and respond empathetically. This limitation leads to suboptimal user experiences and missed opportunities for personalization. To address this gap, this paper presents a groundbreaking prototype of a future voice assistant equipped with the capability to recognize and respond to users’ emotional states. The paper outlines the architecture of a speech-emotion recognition system that utilizes a deep neural network incorporating a sparse attention mechanism, yielding promising results. Furthermore, the paper embarks on an initial exploration of strategies for emotion-based responses, drawing inspiration from human techniques for managing unpleasant emotions. A user study is conducted, exposing participants to emotions such as anger, sadness, and fear, while engaging the voice assistant to elicit happiness. Preliminary findings suggest a tendency toward neutral emotional responses, providing valuable insights into the potential of speech feature analysis as an alternative approach for identifying and responding to emotions.