Real-time Dynamic Sign Recognition using MediaPipe
Youssef Farhan, Abdessalam Aït Madi
Abstract
People live in societies, so communication is necessary to express their needs and ideas. Deaf-mute people use sign language as their means of communication, but most people are unaware of its meaning, so creating an effective system to help them communicate is important. This paper proposes an American Sign Language (ASL) recognition system for 12 dynamic signs in real-time using the MediaPipe framework and a Long Short-Term Memory (LSTM) network. To improve system performance, only the relevant features were extracted from the dynamic sign video, and two new useful features were generated (angles and distances). The proposed system achieved a test accuracy of 97,2%.
Topics & Concepts
Computer scienceSign (mathematics)Sign languageMeaning (existential)Long short term memoryTerm (time)Sign systemSpeech recognitionAmerican Sign LanguageArtificial intelligenceHuman–computer interactionComputer visionNatural language processingRecurrent neural networkArtificial neural networkLinguisticsCommunicationPsychologyMathematical analysisQuantum mechanicsPhysicsMathematicsPhilosophyPsychotherapistHand Gesture Recognition SystemsHearing Impairment and CommunicationHuman Pose and Action Recognition