Shortformer: Better Language Modeling using Shorter Inputs
Ofir Press, Noah A. Smith, Mike Lewis
Abstract
Ofir Press, Noah A. Smith, Mike Lewis. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Topics & Concepts
PerplexityLanguage modelTransformerComputer sciencePosition (finance)AlgorithmWord (group theory)Artificial intelligenceMathematicsEngineeringVoltageFinanceGeometryEconomicsElectrical engineeringTopic ModelingNatural Language Processing TechniquesText Readability and Simplification