Evaluating ChatGPT and GPT-4 for Visual Programming
Adish Singla
Abstract
Generative AI has the potential to drastically improve the landscape of computing education by automatically generating personalized feedback and content. In particular, this potential lies in the advanced capabilities of state-of-the-art deep generative and large language models such as OpenAI’s Codex [7], ChatGPT [11], and GPT-4 [12]. In our work, we seek to investigate the capabilities of these models in visual programming domains popularly used for K-8 programming education, including domains like Scratch [17], Hour of Code: Maze Challenge by Code.org [4, 5], and Karel [13].
Topics & Concepts
Computer scienceGenerative grammarKarelProgramming languageScratchCode (set theory)Visual programming languageArtificial intelligenceProgramming paradigmState (computer science)Software engineeringSet (abstract data type)Teaching and Learning ProgrammingEducational Games and GamificationSoftware Testing and Debugging Techniques