Train Big, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers
Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez
2020International Conference on Machine Learning111 citations
Topics & Concepts
InferenceComputer scienceTransformerTraining (meteorology)Artificial intelligenceMachine learningEngineeringElectrical engineeringVoltagePhysicsMeteorologyNeural Networks and ApplicationsMachine Learning and Data ClassificationTime Series Analysis and Forecasting