Big code != big vocabulary
Rafael-Michael Karampatsis, Hlib Babii, Romain Robbes, Charles Sutton, Andrea Janes
Abstract
Statistical language modeling techniques have successfully been applied to large source code corpora, yielding a variety of new software development tools, such as tools for code suggestion, improving readability, and API migration. A major issue with these techniques is that code introduces new vocabulary at a far higher rate than natural language, as new identifier names proliferate. Both large vocabularies and out-of-vocabulary issues severely affect Neural Language Models (NLMs) of source code, degrading their performance and rendering them unable to scale.
Topics & Concepts
Computer scienceReadabilityVocabularyIdentifierSource codeRendering (computer graphics)Natural languageNatural language processingArtificial intelligenceCode (set theory)Programming languageLinguisticsSet (abstract data type)PhilosophySoftware Engineering ResearchTopic ModelingNatural Language Processing Techniques