Assessing AI-Based Code Assistants in Method Generation Tasks
Vincenzo Corso, Leonardo Mariani, Daniela Micucci, Oliviero Riganelli
Abstract
AI-based code assistants are increasingly popular as a means to enhance productivity and improve code quality. This study compares four AI-based code assistants, GitHub Copilot, Tabnine, ChatGPT, and Google Bard, in method generation tasks, assessing their ability to produce accurate, correct, and efficient code. Results show that code assistants are useful, with complementary capabilities, although they rarely generate ready-to-use correct code.
Topics & Concepts
Computer scienceCode (set theory)Code reviewCode generationProgramming languageQuality (philosophy)Program codeSource codeProductivitySoftware engineeringStatic program analysisSoftwareComputer securitySoftware developmentMacroeconomicsEconomicsKey (lock)Set (abstract data type)EpistemologyPhilosophySoftware Engineering ResearchSoftware Engineering Techniques and PracticesSoftware Reliability and Analysis Research