Efficient Asynchronous Vertical Federated Learning via Gradient Prediction and Double-End Sparse Compression
Ming Li, Yiwei Chen, Yiqin Wang, Yunhe Pan
Abstract
Vertical federated learning is a subset of federated learning whose training dataset is vertically distributed among the federations. However, as a natural synchronous algorithm, classical vertical federated learning suffers from “Liebig's Law”. In this paper, we propose a novel asynchronous vertical federated learning framework with gradient prediction and double-end sparse compression to accelerate the training process and reduce the intermediate result transmission. Our simulation results show that our framework can achieve 65.41% training acceleration and 86.90% traffic volume reduction at no cost of accuracy compared with the classical framework.
Topics & Concepts
Computer scienceAsynchronous communicationProcess (computing)Reduction (mathematics)Compression (physics)AccelerationEnd-to-end principleFederated learningTransmission (telecommunications)Training (meteorology)Artificial intelligenceMachine learningComputer networkMathematicsComposite materialPhysicsGeometryClassical mechanicsOperating systemMaterials scienceMeteorologyTelecommunicationsPrivacy-Preserving Technologies in DataInternet Traffic Analysis and Secure E-votingStochastic Gradient Optimization Techniques