Adaptive Sparsification and Quantization for Enhanced Energy Efficiency in Federated Learning
Ouiame Marnissi, Hajar El Hammouti, El Houcine Bergou
Abstract
Federated learning is a distributed learning framework that operates effectively over wireless networks. It enables devices to collaboratively train a model over wireless links by sharing model parameters rather than personal data. However, a key challenge in federated learning arises from the limited computational and communication resources of the devices. Therefore, optimizing energy consumption is crucial for practical implementations of federated learning. In this context, we address energy minimization by applying compression techniques that reduce the number of bits required for both local computation and uplink communications. We develop an optimization framework that aims to minimize the total energy consumption across all devices involved in the training process. This framework considers quantization levels for local computation and uplink transmission, as well as the level of sparsification for parameter transmission. The optimization is constrained by requirements on latency and the target accuracy. To solve this complex problem, we first derive the required number of global training rounds, to achieve the desired accuracy. We then employ an iterative algorithm to efficiently find the optimal parameters of the studied problem. Our numerical results show that the proposed approach achieves significant performance and considerably reduces the energy consumption compared to two different federated learning baseline schemes.