A Transfer Learning-Based Framework for Efficient American Sign Language Recognition
Mohammad Shahin Shah, Mohammad Abdul Kader, Kanij Fatema Sworna, Md. Ibrahim Shakib, Md. Sorowar Mahabub Rabby, Md. Khaliluzzaman
Abstract
The deaf community faces communication barriers due to their use of sign language, which significantly impacts their daily lives. These barriers arise from limited access to sign language interpreters and reliance on lipreading or writing. A groundbreaking solution using Convolutional Neural Networks (CNNs) is being explored to address this issue. Our study focuses on applying CNN based transfer learning models for American Sign Language recognition (ASL). In this regard, in this work, propose the framework where transfer learning backbones are used to capture the spatial features inherent in ASL gestures, and the final convolutional layers feature maps are converted to the global average pooling (GAP) layers. After that, the GAP layers are connected to the output layers. Finally, the Softmax classifier is used to recognize the sign language. From the experiment, it is revealed that the VGG19 model achieved 99.99 % accuracy in sign language recognition, while VGG16 and ResNet50 achieved 99.94 % and 99.81% accuracy respectively, representing a significant advancement. Our work outperforms existing cutting-edge models in American Sign Language recognition, attaining exceptional accuracy and setting new standards in the area. This research opens the door to a future where communication barriers are overcome, allowing everyone to connect.