Communication-efficient Federated Learning Through 1-Bit Compressive Sensing and Analog Aggregation
Xin Fan, Yue Wang, Yan Huo, Zhi Tian
Abstract
This paper studies communication-efficient federated learning (FL) over the air, which is based on 1-bit compressive sensing (CS) and analog aggregation transmissions. To analyze the impact of these two communication efficiency oriented technologies on FL, we derive a closed-form expression for the expected convergence rate of the FL algorithm. Our theoretical result implies that the communication efficiency comes at the expense of the performance degradation due to the aggregation errors caused by sparsification, dimension reduction, quantization, signal reconstruction and noise. Then, we formulate a joint optimization problem to mitigate the impact of these aggregation errors on FL by an optimal scheduling and power scaling policy. An enumerated method is proposed to solve this non-convex problem, which is optimal but becomes computationally infeasible as the number of devices increases. Hence, we further propose a suboptimal solution based on the alternating direction method of multiplier to reduce the complexity when applied in large-scale networks. Simulation results show that our proposed 1-bit CS based FL over the air achieves comparable performance to the ideal case where conventional FL without compression and quantification is applied over error-free aggregation, at much reduced communication overhead and transmission latency.