Energy-Efficient Resource Management for Federated Edge Learning With CPU-GPU Heterogeneous Computing
Qunsong Zeng, Yuqing Du, Kaibin Huang, Kin K. Leung
Abstract
Edge machine learning involves the deployment of learning algorithms at the network edge to leverage massive distributed data and computation resources to train <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">artificial intelligence</i> (AI) models. Among others, the framework of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">federated edge learning</i> (FEEL) is popular for its data-privacy preservation. FEEL coordinates global model training at an edge server and local model training at devices that are connected by wireless links. This work contributes to the energy-efficient implementation of FEEL in wireless networks by designing joint <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">computation-and-communication resource management</i> ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {C}^{2}$ </tex-math></inline-formula> RM). The design targets the state-of-the-art heterogeneous mobile architecture where parallel computing using both CPU and GPU, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">heterogeneous computing</i> , can significantly improve both the performance and energy efficiency. To minimize the sum energy consumption of devices, we propose a novel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {C}^{2}$ </tex-math></inline-formula> RM framework featuring multi-dimensional control including bandwidth allocation, CPU-GPU workload partitioning and speed scaling at each device, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {C}^{2}$ </tex-math></inline-formula> time division for each link. The key component of the framework is a set of equilibriums in energy rates with respect to different control variables that are proved to exist among devices or between processing units at each device. The results are applied to designing efficient algorithms for computing the optimal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {C}^{2}$ </tex-math></inline-formula> RM policies faster than the standard optimization tools. Based on the equilibriums, we further design energy-efficient schemes for device scheduling and greedy spectrum sharing that scavenges “spectrum holes” resulting from heterogeneous <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {C}^{2}$ </tex-math></inline-formula> time divisions among devices. Using a real dataset, experiments are conducted to demonstrate the effectiveness of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathrm {C}^{2}$ </tex-math></inline-formula> RM on improving the energy efficiency of a FEEL system.