A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician
Candy Obdulia Sosa-Jiménez, Homero Vladimir Ríos-Figueroa, Ana Luisa Solís-González-Cosío
Abstract
Few hearing people know and use Mexican Sign Language (MSL). Consequently, this is the main barrier between deaf and hearing people. This study proposes a system that recognizes and animates in real time a set of signs belonging to the semantic field of a general medicine consultation service. Therefore, a linkage between a hearing doctor and a deaf patient can be established in a non-intrusive way and with an easy dynamic interaction. Our main contribution is a bidirectional translator system for Mexican sign language in the context of a general medical consultation. The recognition module uses a MS Kinect sensor to obtain the sign trajectories, and images to feed hidden Markov Models (HMMs) for processing signs samples in real time. The experiments show recognition of 82 different signs from 22 participants. As a result, accuracy and F1 scores average rates of 99% and 88% were obtained.