How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych
Abstract
Phillip Rust, Jonas Pfeiffer, Ivan Vulić, Sebastian Ruder, Iryna Gurevych. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Topics & Concepts
Computer scienceTask (project management)Set (abstract data type)VocabularyMultilingualismNatural language processingRepresentation (politics)Downstream (manufacturing)Artificial intelligenceLinguisticsPolitical scienceLawPoliticsProgramming languageEconomicsOperations managementManagementPhilosophyTopic ModelingNatural Language Processing TechniquesText Readability and Simplification