Identifying Incorrect Labels in the CoNLL-2003 Corpus
Frederick Reiss, Hong Xu, Bryan Cutler, Karthik Muthuraman, Zachary Eichenberger
Abstract
The CoNLL-2003 corpus for Englishlanguage named entity recognition (NER) is one of the most influential corpora for NER model research. A large number of publications, including many landmark works, have used this corpus as a source of ground truth for NER tasks. In this paper, we examine this corpus and identify over 1300 incorrect labels (out of 35089 in the corpus). In particular, the number of incorrect labels in the test fold is comparable to the number of errors that state-of-the-art models make when running inference over this corpus.
Topics & Concepts
Computer scienceInferenceNatural language processingArtificial intelligenceGround truthNamed-entity recognitionText corpusLandmarkCorpus linguisticsEconomicsTask (project management)ManagementNatural Language Processing TechniquesTopic ModelingText Readability and Simplification