Communication-Efficient Federated Learning via Dynamic Sparsity: An Adaptive Pruning Ratio Based on Weight Importance

Abstract

In Federated Learning (FL), optimizing communication efficiency and utilizing computational resources present key challenges. This paper proposes the Federated Adaptive Deep Sparse Representation (FedADSR) method. FedADSR improves model training efficiency by adjusting the pruning ratio based on the importance of model weights, significantly reducing communication overhead. Traditional fixed pruning methods have limitations. FedADSR employs an importance-based threshold mechanism to dynamically adjust the pruning ratio for each layer, offering a more efficient and adaptive pruning solution. This reduces the computational burden on client devices and enhances system scalability. Consumer electronics, such as smartphones, smart home devices, and wearable technology, particularly benefit from this approach.To validate the effectiveness of FedADSR, we conducted experiments across six datasets: MNIST, FMNIST, CIFAR-10, CIFAR-100, SVHN, and Tiny-ImageNet. The experimental results demonstrate that FedADSR significantly reduces communication overhead while maintaining competitive accuracy levels compared to existing methods such as DSFL, FedLion, and PruneFL.Specifically, the accuracy of FedADSR is not lower than that of other state-of-the-art algorithms across all six datasets, and the communication overhead is reduced by an average of 25% across all scenarios. These findings highlight the efficiency of FedADSR in balancing model sparsity and performance. This adaptive pruning strategy enables optimal resource utilization between client devices and central servers, offering a practical solution for real-world federated learning scenarios, particularly in consumer electronics where low latency and energy efficiency are crucial.

Topics & Concepts