VisionARy: Exploratory research on Contextual Language Learning using AR glasses with ChatGPT
Hyungmin Lee, Chen-Chun Hsia, Aleksandr Tsoy, Sung-Min Choi, Hanchao Hou, Shiguang Ni
Abstract
Language learning is a challenging and time-consuming process, requiring an immersive environment and active usage of the language. However, English as a Foreign Language (EFL) learners often face difficulties in obtaining immersive learning experiences due to one-sided teaching and a lack of real-life English language contexts for practice. To overcome this challenge, various approaches have been proposed that leverage technologies such as mobile-based augmented reality and chatbots. However, these approaches have limitations that hinder full immersion into the language learning environment and fail to adequately contextualize the learning experience. With the recent breakthroughs in large language models and the development of lighter and more powerful AR glasses, we propose VisionARy, the first system that integrates ChatGPT into AR glasses to improve oral language skills and provide a contextualized learning experience. The evaluation results suggest that VisionARy has the potential to be effective and is highly accepted in improving oral language skills, compared to traditional learning methods. Furthermore, our findings provide important insights for the future design of language learning systems.