Do Transformer Modifications Transfer Across Implementations and Applications?
Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Févry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam P. Roberts, Colin Raffel
2021Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing80 citationsDOIOpen Access PDF
Abstract
Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.
Topics & Concepts
ImplementationComputer scienceTransformerArtificial intelligenceProgramming languageElectrical engineeringVoltageEngineeringNatural Language Processing TechniquesTopic ModelingMultimodal Machine Learning Applications