Enhancing language learning through generative AI feedback on picture-cued writing tasks
Yipeng Zhuang, Ruibin Zhao, Zhiwei Xie, Philip L. H. Yu
Abstract
Generative AI (GAI) is transforming education, despite its widespread use, its role in supporting student learning remains underexplored, and how to effectively leverage GAI for educational purposes is still unclear. This study focuses on enhancing language learning through GAI, specifically in picture-cued writing tasks, where students describe life scenarios depicted in pictures through text. A Generative AI-assisted language learning system was developed, powered by fine-tuned multimodal Large Language Models (LLMs), designed to evaluate students' textual descriptions in relation to corresponding images and provide adaptive feedback. A lot of middle school students participated in the experiments, generating more than 5000 writing samples assessed by the system. The results demonstrate the effectiveness of this approach, with performance metrics indicating a significant improvement in writing scores. Students reported high levels of engagement and perceived usefulness of the system. These findings highlight the pedagogical value of adaptive GAI feedback, its potential for scalable classroom integration, and offer a dynamic and adaptive assessment method that enhances students’ engagement and performance in language learning. • Generative AI improves language learning through real-life visual description tasks with multimodal LLMs. • Feasible to provide automated, personalized feedback for self-directed language learning with Generative AI. • Automated feedback improved writing quality and motivated students in language learning.