Are Pretrained Convolutions Better than Pretrained Transformers?
Yi Tay, Mostafa Dehghani, Jai Prakash Gupta, Vamsi Aribandi, Dara Bahri, Zhen Qin, Donald Metzler
Abstract
Yi Tay, Mostafa Dehghani, Jai Prakash Gupta, Vamsi Aribandi, Dara Bahri, Zhen Qin, Donald Metzler. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Topics & Concepts
Computer scienceTransformerNatural language processingVolume (thermodynamics)Artificial intelligenceEngineeringPhysicsElectrical engineeringQuantum mechanicsVoltageTopic ModelingSentiment Analysis and Opinion MiningHate Speech and Cyberbullying Detection