Dice Loss for Data-imbalanced NLP Tasks
Xiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang, Fei Wu, Jiwei Li
Abstract
Many NLP tasks such as tagging and machine reading comprehension (MRC) are faced with the severe data imbalance issue: negative examples significantly outnumber positive ones, and the huge number of easy-negative examples overwhelms training. The most commonly used cross entropy criteria is actually accuracy-oriented, which creates a discrepancy between training and test. At training time, each training instance contributes equally to the objective function, while at test time F1 score concerns more about positive examples.
Topics & Concepts
DiceComputer scienceArtificial intelligenceNatural language processingMachine learningTask (project management)Cross entropySupport vector machineFalse positive paradoxParaphraseTraining setTest dataPrinciple of maximum entropyStatisticsMathematicsManagementProgramming languageEconomicsTopic ModelingNatural Language Processing TechniquesText Readability and Simplification