Word Discovery in Visually Grounded, Self-Supervised Speech Models
Puyuan Peng, David Harwath
Abstract
OursFigure 1: HuBERT: sum of attention weights each frame receives from other frames.Ours (VG-HuBERT3): attention weights each frame receives from the [CLS A] token.Attention weights from different attention heads are coded with different colors.
Topics & Concepts
Computer scienceWord (group theory)Natural language processingSpeech recognitionArtificial intelligenceLinguisticsPhilosophyMultimodal Machine Learning ApplicationsSpeech and dialogue systemsNatural Language Processing Techniques