iobes: Library for Span Level Processing

Abstract

Many tasks in natural language processing, such as named entity recognition and slotfilling, involve identifying and labeling specific spans of text. In order to leverage common models, these tasks are often recast as sequence labeling tasks. Each token is given a label and these labels are prefixed with special tokens such as B-or I-. After a model assigns labels to each token, these prefixes are used to group the tokens into spans.

Topics & Concepts