AR Computer-Assisted Learning for Children with ASD based on Hand Gesture and Voice Interaction
Kahina Amara, Chahrazed Boudjemila, Nadia Zenati, Oualid Djekoune, Drifa Aklil, Mouna Kenoui
Abstract
For children with autism spectrum disorder (ASD), innovative computer-assisted solutions can provide useful teaching, learning, and support tools. Speech, video, narrative, and a mix of 2D/3D images of virtual objects shown on screens and mobile phones can help to improve the learning experience. Augmented Reality (AR) presents a valuable technology for achieving certain educational objectives. This study presents a gesture and voice-based learning framework for children with ASD that offers interactivity, engagement, and visual support during school/cognitive therapy sessions. The proposed prototype was designed to increase children's engagement and focus during the session and achieve a high level of social interaction and inclusion. It was implemented in accordance with the requirements of specialists from regional rehabilitation centres for children with ASD and structured activities and AR-based games were defined. A multi-baseline feasibility pilot study was carried out with 18 children, aged 2–12 years, diagnosed with ASD and with learning difficulties (concentration and language development problems, social interaction and inclusion), and 3 therapists/parents. Preliminary qualitative and quantitative results show significantly higher engagement and concentration time using the proposed AR-computer system compared to a non-computer condition. In addition to improved receptive vocabulary and social interaction for the children, the prototype reduces teacher workload while engaging the children.