Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Vuong The Tran, Dani Yogatama, Donald Metzler
Abstract
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Tran, Dani Yogatama, Donald Metzler. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.
Topics & Concepts
ScalingScaling lawComputer scienceInductive biasStatistical physicsMathematicsPhysicsEngineeringGeometrySystems engineeringMulti-task learningTask (project management)Natural Language Processing Techniques