ETC: Encoding Long and Structured Inputs in Transformers
Joshua Ainslie, Santiago Ontañón, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Yang Li
Abstract
Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Li Yang. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.
Topics & Concepts
TransformerComputer scienceEncoding (memory)Natural language processingArt historyArtificial intelligenceHistoryEngineeringElectrical engineeringVoltageTopic ModelingNatural Language Processing TechniquesAdvanced Text Analysis Techniques