The EarlyBIRD Catches the Bug: On Exploiting Early Layers of Encoder Models for More Efficient Code Classification
Anastasiia Grishina, Max Hort, Leon Moonen
Abstract
The use of modern Natural Language Processing (NLP) techniques has shown to be beneficial for software engineering tasks, such as vulnerability detection and type inference. However, training deep NLP models requires significant computational resources. This paper explores techniques that aim at achieving the best usage of resources and available information in these models.
Topics & Concepts
Computer scienceCode (set theory)EncoderSoftware bugProgramming languageSoftwareOperating systemSet (abstract data type)Software Engineering ResearchTopic ModelingNatural Language Processing Techniques