Harder Task Needs More Experts: Dynamic Routing in MoE Models
Quzhe Huang, Zhenwei An, Nan Zhuang, Mingxu Tao, Chen Zhang, Jin Yang, Kun Xu, Kun Xu, Liwei Chen, Songfang Huang, Yansong Feng
Abstract
Quzhe Huang, Zhenwei An, Nan Zhuang, Mingxu Tao, Chen Zhang, Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Songfang Huang, Yansong Feng. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
Topics & Concepts
Task (project management)Computer scienceRouting (electronic design automation)Adaptive routingStatic routingSystems engineeringEngineeringEmbedded systemRouting protocolSemantic Web and OntologiesService-Oriented Architecture and Web Services