A Multi-party Conversational Social Robot Using LLMs
Angus Addlesee, Neeraj Cherakara, Nivan Nelson, Daniel Hernández García, Nancie Gunson, Weronika Sieińska, Marta Romeo, Christian Dondrup, Oliver Lemon
Abstract
In this paper, we describe our setting and the architecture of our LLM-based dialogue system embodied in a social robot and able to have multi-party conversations. Each component is detailed, and a video of the full system is available with the appropriate components highlighted in real-time. Our system decides when it should take its turn, generates human-like clarification requests when the patient pauses mid-utterance, answers in-domain questions (grounding to the in-prompt knowledge), and responds appropriately to out-of-domain requests (like generating jokes or quizzes).
Topics & Concepts
UtteranceComponent (thermodynamics)Embodied cognitionComputer scienceArchitectureRobotDomain (mathematical analysis)Human–computer interactionDialog systemSocial robotArtificial intelligenceWorld Wide WebRobot controlMobile robotDialog boxThermodynamicsMathematicsArtVisual artsMathematical analysisPhysicsAI in Service InteractionsSpeech and dialogue systemsSocial Robot Interaction and HRI