Omniforce: on human-centered, large model empowered and cloud-edge collaborative AutoML system
Chao Xue, Wei Liu, Shuai Xie, Zhenfang Wang, Jiaxing Li, Xuyang Peng, Liang Ding, Shanshan Zhao, Qiong Cao, Yibo Yang, Fengxiang He, Bohua Cai, Rongcheng Bian, Yiyan Zhao, Heliang Zheng, Xiangyang Liu, Dongkai Liu, Daqing Liu, Li Shen, Chang Li, Shijin Zhang, Fei Wang, Yinhao Bai, Yukang Zhang, Guanpu Chen, Shixiang Chen, Jing Zhang, Yibing Zhan, Chaoyue Wang, Dacheng Tao
Abstract
Addressing the open-environment issue with pure data-driven approaches, especially for large models (LM) that require great efforts for data curation and mix, training recipes, and collaboration with small models, makes current Automated machine learning (AutoML) systems inefficient and computationally intractable. We introduce OmniForce (OF), a human-centered AutoML system that yields both human-assisted ML and ML-assisted human techniques, putting an AutoML system into practice and building AI in open-environment scenarios, such as using large models in industrial supply chains. Concretely, we present OmniForce in terms of management for data, search space, and algorithms; pipeline-driven development and deployment collaborations; and widely provisioned and crowd-sourcing application algorithms. OmniForce can be run either on a public/private cloud or in an on-premise environment, and the models constructed by OmniForce can be quickly and automatically turned into remote services, dubbed model as a service (MaaS). Experimental results obtained demonstrate the efficacy and efficiency of OmniForce.