Does generative AI help in learning programming: Students’ perceptions, reported use and relation to performance
Marina Lepp, Joosep Kaimre
Abstract
In 2022, the release of ChatGPT marked a significant advancement in the use of Artificial Intelligence (AI) chatbots, particularly impacting fields like computer science and education. The ability to generate code snippets using AI chatbots has introduced new opportunities and challenges in teaching programming. However, there is limited agreement on how students integrate them into their learning processes. This study aims to explore how students utilize AI chatbots in the "Object-Oriented Programming" course and examine the relationship between chatbot usage and academic performance. To address this, 231 students completed a survey assessing the frequency and manner of chatbot usage. Descriptive statistical methods were employed to analyze usage and perceptions, while Spearman’s correlation was used to investigate the connection between chatbot usage and course performance. Results indicated that students primarily relied on AI chatbots for programming tasks. Interestingly, students’ performance negatively correlates with the reported frequency of using these tools. These findings provide valuable insights for programming educators, offering a better understanding of students' perceptions and use of AI chatbots. This knowledge can inform strategies for integrating these tools effectively into computer science education. • Most students used AI for coding tasks, but few used it weekly, showing no dependency yet. • Few students feared plagiarism, suggesting students don’t link AI use to it—raising academic concerns. • Frequent AI use links to lower scores, likely as weaker students rely more on study aids. • New uses like test data generation and code translation can guide AI integration into CS education.