Effectiveness of Generative AI in Automated Written Corrective Feedback With Prompting
Jiahui Wu, Jianwei Li, Zi-Gang Ge, Mingrui Xu, Lin Li, Ru Zhang
Abstract
Automated written corrective feedback (AWCF) tools play a crucial role in supporting English writing instruction. However, issues such as insufficient accuracy and hallucination have undermined users’ trust in these systems. To address these challenges, this study investigates the potential of Generative Artificial Intelligence (GAI) enhanced by prompting. Specifically, we evaluate the performance of several GAI models, including GPT-4, in providing written corrective feedback compared to commercial AWCF tools and other advanced models. The study adopts a dual-method approach: (1) a comprehensive model evaluation using established English writing datasets to assess error correction performance via various prompting strategies, and (2) an empirical study involving teachers and students to examine the system’s practical efficacy and users’ perspective. Quantitative results indicate that GPT-4 with chain-of-thought prompting significantly outperforms commercial tools, achieving improved consistency and accuracy in error detection and correction. Qualitative feedback from participants further supports the system’s potential to enhance students’ writing quality and confidence, while concurrently reducing teachers’ workload and optimizing instructional efficiency, despite concerns regarding occasional overcorrection. These findings emphasize the benefits of task-specific prompting in GAI-based AWCF systems and provide actionable insights for integrating advanced AI feedback into educational practices.