Transcending Scaling Laws with 0.1% Extra Compute
Yi Tay, Jason Wei, Hyung Won Chung, Vinh Vuong The Tran, David R. So, Siamak Shakeri, Xavier García, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V. Le, Mostafa Dehghani
Abstract
Yi Tay, Jason Wei, Hyung Chung, Vinh Tran, David So, Siamak Shakeri, Xavier Garcia, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc Le, Mostafa Dehghani. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
Topics & Concepts
Scaling lawGarciaPhilosophyLawScalingComputer scienceHumanitiesMathematicsPolitical scienceGeometryTopic ModelingNatural Language Processing TechniquesSpeech Recognition and Synthesis