HoloSound: Combining Speech and Sound Identification for Deaf or Hard of Hearing Users on a Head-mounted Display
Ru Guo, Yiru Yang, Johnson Kuang, Bin Xue, Dhruv Jain, Steven M. Goodman, Leah Findlater, Jon E. Froehlich
Abstract
Head-mounted displays can provide private and glanceable speech and sound feedback to deaf and hard of hearing people, yet prior systems have largely focused on speech transcription. We introduce HoloSound, a HoloLens-based augmented reality (AR) prototype that uses deep learning to classify and visualize sound identity and location in addition to providing speech transcription. This poster paper presents a working proof-of-concept prototype, and discusses future opportunities for advancing AR-based sound awareness.
Topics & Concepts
Computer scienceTranscription (linguistics)Speech recognitionSpeech soundAugmented realitySound (geography)Identification (biology)Identity (music)Head (geology)Human–computer interactionAcousticsLinguisticsGeologyBotanyBiologyPhilosophyPhysicsGeomorphologySpeech and Audio ProcessingHearing Loss and RehabilitationMusic and Audio Processing