Leveraging Process-Action Epistemic Network Analysis to Illuminate Student Self-Regulated Learning with a Socratic Chatbot
Joel Weijia Lai, Wei Qiu, Muang Thway, Lei Zhang, Nurabidah Binti Jamil, Chit Lin Su, Samuel Soo Hwee Ng, Fun Siong Lim
Abstract
The growing use of generative AI (GenAI) has sparked discussions regarding integrating these tools into educational settings to enrich the learning experience of teachers and students. Self-regulated learning (SRL) research is pivotal in addressing this inquiry. One prevalent manifestation of GenAI is the large-language model (LLM) chatbot, enabling users to seek information and assistance. This paper aims to showcase how data on student interaction with a chatbot can be used in learning analytics to gain insights into SRL. This is achieved by adapting existing SRL frameworks to comprehend 34 students’ interaction with an educational Socratic chatbot for a statistics class at the introductory undergraduate level. Chatbot conversations from students are categorized into learning actions and processes using the framework’s process-action library. Thereafter, we analyze this data through ordered epistemic network analysis, furnishing valuable insights into how different students interact with the chatbot. Our findings reveal that higher-scoring students engage more frequently in reflective and evaluative activities, while lower-scoring students focus on searching for answers. Furthermore, students should shift from structured problem-solving, such as solving classroom questions, to questioning fundamental concepts with the chatbot and soliciting more examples to improve their learning gains.