Transfer learning with YOLOV8 for real-time recognition system of American Sign Language Alphabet
Bader Alsharif, Easa Alalwany, Mohammad Ilyas
Abstract
Sign language serves as a sophisticated means of communication vital to individuals who are deaf or hard of hearing, relying on hand movements, facial expressions, and body language to convey nuanced meaning. American Sign Language (ASL) exemplifies this linguistic complexity with its distinct grammar and syntax. The advancement of real-time ASL gesture recognition has explored diverse methodologies, including motion sensors and computer vision techniques. This study specifically addresses the recognition of ASL alphabet gestures using computer vision through Mediapipe for hand movement tracking and YOLOv8 for training the deep learning model. The model achieved notable performance metrics: precision of 98%, recall rate of 98%, F1 score of 99%, mean Average Precision (mAP) of 98%, and mAP50-95 of 93%, underscoring its exceptional accuracy and sturdy capabilities. • YOLOv8 to interpret the American Sign Language alphabet into text in real-time. • YOLOv8 outperformed all previously reported findings in recognizing American Sign Language alphabet hand gestures.