On the Reliability of the Area Under the ROC Curve in Empirical Software Engineering
Luigi Lavazza, Sandro Morasca, Gabriele Rotoloni
Abstract
Binary classifiers are commonly used in software engineering research to estimate several software qualities, e.g., defectiveness or vulnerability. Thus, it is important to adequately evaluate how well binary classifiers perform, before they are used in practice. The Area Under the Curve (AUC) of Receiver Operating Characteristic curves has often been used to this end. However, AUC has been the target of some criticisms, so it is necessary to evaluate under what conditions and to what extent AUC can be a reliable performance metric.
Topics & Concepts
Receiver operating characteristicMetric (unit)Reliability (semiconductor)Reliability engineeringComputer scienceBinary numberSoftware qualitySoftware metricSoftwareLearning curveVulnerability (computing)Binary classificationData miningMachine learningMathematicsSoftware developmentEngineeringSupport vector machineOperating systemComputer securityPower (physics)Quantum mechanicsArithmeticPhysicsOperations managementSoftware Engineering ResearchSoftware Reliability and Analysis ResearchImbalanced Data Classification Techniques