Token Dropping for Efficient BERT Pretraining
Le Hou, Richard Yuanzhe Pang, Tianyi Zhou, Yuexin Wu, Xinying Song, Xiaodan Song, Denny Zhou
2022Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)16 citationsDOIOpen Access PDF
Abstract
Le Hou, Richard Yuanzhe Pang, Tianyi Zhou, Yuexin Wu, Xinying Song, Xiaodan Song, Denny Zhou. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022.
Topics & Concepts
Security tokenComputer scienceVolume (thermodynamics)Programming languageOperating systemPhysicsQuantum mechanicsNatural Language Processing TechniquesSoftware Testing and Debugging Techniques