Speech2Properties2Gestures
Taras Kucherenko, Rajmund Nagy, Patrik Jonell, Michael Neff, Hedvig Kjellström, Gustav Eje Henter
Abstract
We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture, followed by a prediction of the gesture properties. Those properties are then used as conditioning for a modern probabilistic gesture-generation model capable of high-quality output. This empowers the approach to generate gestures that are both diverse and representational. Follow-ups and more information can be found on the project page: https://svito-zar.github.io/speech2properties2gestures/
Topics & Concepts
GestureComputer scienceGesture recognitionProbabilistic logicArtificial intelligenceNatural language processingHuman–computer interactionSpeech and dialogue systemsNatural Language Processing TechniquesHand Gesture Recognition Systems