Joint Computation Offloading and Resource Allocation in Multi-Edge Smart Communities With Personalized Federated Deep Reinforcement Learning
Zheyi Chen, Bing Xiong, Xing Chen, Geyong Min, Jie Li
Abstract
Through deploying computing resources at the network edge, Mobile Edge Computing (MEC) alleviates the contradiction between the high requirements of intelligent mobile applications and the limited capacities of mobile End Devices (EDs) in smart communities. However, existing solutions of computation offloading and resource allocation commonly rely on prior knowledge or centralized decision-making, which cannot adapt to dynamic MEC environments with changeable system states and personalized user demands, resulting in degraded Quality-of-Service (QoS) and excessive system overheads. To address this important challenge, we propose a novel Personalized Federated deep Reinforcement learning based computation Offloading and resource Allocation method (PFR-OA). This innovative PFR-OA considers the personalized demands in smart communities when generating proper policies of computation offloading and resource allocation. To relieve the negative impact of local updates on global model convergence, we design a new proximal term to improve the manner of only optimizing local Q-value loss functions in classic reinforcement learning. Moreover, we develop a new partial-greedy based participant selection mechanism to reduce the complexity of federated aggregation while endowing sufficient exploration. Using real-world system settings and testbed, extensive experiments demonstrate the effectiveness of the PFR-OA. Compared to benchmark methods, the PFR-OA achieves better trade-offs between delay and energy consumption and higher task execution success rates under different scenarios.