Large-Scale Computation Offloading Using a Multi-Agent Reinforcement Learning in Heterogeneous Multi-Access Edge Computing
Zhen Gao, Lei Yang, Yu Dai
Abstract
Recently, existing computation offloading methods have provided extremely low service latency for mobile users (MUs) in multi-access edge computing (MEC). However, this remains a challenge in large-scale mixed cooperative-competitive MUs heterogeneous MEC environments. Moreover, existing methods focus more on all offloaded tasks handled by static resource allocation MEC servers (ESs) within a time interval, ignoring on-demand requirements of heterogeneous tasks, resulting in many tasks being dropped or wasting resources, especially for latency-sensitive tasks. To address these issues, we present a decentralized computation offloading solution based on the Attention-weighted Recurrent Multi-Agent Actor-Critic (ARMAAC). First, we design a recurrent actor-critic framework to assist MU agents in remembering historical resource allocation information of ESs to better understand the future state of ESs, especially in dynamic resource allocation. Second, an attention mechanism is introduced to compress the joint observation space dimension of all MUs agent to adapt to large-scale MUs. Finally, the actor-critic framework with double centralized critics and Dueling network is redesigned considering the instability and convergence difficulties caused by the sensitive relationship between the actor and critic networks. The experiments show that ARMAAC improves task completion rates and reduces average system cost by 11.01% <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 14.03% and 10.45% <inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 15.56% compared with baselines.