Summarise, elaborate, try again: exploring the effect of feedback literacy on AI-enhanced essay writing
Brendan E. Hawkins, Daniel Taylor-Griffiths, Jason M. Lodge
Abstract
Conservative estimates suggest that since ChatGPT was released in November 2022, more than 70% of university students are using AI in their studies. In particular, students have reported the benefits of using generative AI for real-time, personalised feedback. This study explores how students use generative AI for feedback using the novel methodology of a simulated assessment task. In individual, 1-hour sessions, 32 participants completed a screen-recorded, 25-minute essay writing task. Participants were asked to use generative AI to enhance their work. Following a brief questionnaire, participants completed a video-stimulated interview. While watching the essay screen recording, participants discussed their process, specifically focusing on how AI was used. A thematic analysis of transcribed interviews identified four distinct themes: feed-forward (initial content requests), feedback (requesting assessments of own work), feedback evaluation (making decisions based on AI output, including providing feedback to AI), and AI avoidance (deliberately not using AI). A multiple regression found feedback literacy to be a significant predictor of essay task performance. This exploratory study demonstrates the use of a novel methodology to help understand feedback literacy and evaluative judgement skills as students become equipped for interacting with AI.