Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models
Takanori Ashihara, Takafumi Moriya, Kohei Matsuura, Tomohiro Tanaka
Topics & Concepts
Computer scienceDistillationTask (project management)Artificial intelligenceNatural language processingMachine learningSpeech recognitionEngineeringChemistrySystems engineeringOrganic chemistryIntelligent Tutoring Systems and Adaptive LearningSpeech and dialogue systemsNatural Language Processing Techniques