Fusing Multitask Models by Recursive Least Squares
Xiaobin Li, Lianlei Shan, Weiqiang Wang
Abstract
It is easy to obtain multi-tasking models from open source platforms or various organizations. However, using these models at the same time will bring a great burden on storage and reduce computing efficiency. In this paper, we propose a transformation-based multi-task fusion method, called transformation fusion(TF), which is implemented by recursive least squares. The recursive transformation fusion not only reduces the storage burden brought by models fusion but also avoids computing the inverse matrix of high-dimensional matrix. Our multi-model fusion method can also be applied to many mainstream tasks, such as multi-task learning and offline distributed learning. Our fusion method can be applied to data-based fusion tasks as well as data-free fusion tasks. Extensive experiments demonstrate the effectiveness of our fusion method.