Why only Micro-F1? Class Weighting of Measures for Relation Classification
David Harbecke, Yuxuan Chen, Leonhard Hennig, Christoph Alt
Abstract
Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F 1 , macro-F 1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.
Topics & Concepts
WeightingMacroRelation (database)Computer scienceMeasure (data warehouse)Class (philosophy)Strengths and weaknessesData miningArtificial intelligenceA-weightingMachine learningPattern recognition (psychology)PhilosophyRadiologyEpistemologyMedicineProgramming languageImbalanced Data Classification TechniquesMachine Learning and Data ClassificationAnomaly Detection Techniques and Applications