Multiagent Meta-Reinforcement Learning for Optimized Task Scheduling in Heterogeneous Edge Computing Systems
Liwen Niu, Xianfu Chen, Ning Zhang, Yongdong Zhu, Rui Yin, Celimuge Wu, Yangjie Cao
Abstract
Mobile-edge computing (MEC) brings the potential to address the ever increasing computation demands from the mobile users (MUs). In addition to local processing, the resource-constrained MUs in an MEC system can also offload computation to the nearby servers for remote execution. With the explosive growth of mobile devices, computation offloading faces the challenge of spectrum congestion, which, in turn, deteriorates the overall quality of computation experience. This article, hence, investigates computation task scheduling in a heterogeneous cellular and WiFi MEC system. Such a system provides both licensed and unlicensed spectrum opportunities. Due to the sharing of communication and computation resources as well as the uncertainties, we formulate the problem of computation task scheduling among the competing MUs in a stationary heterogeneous edge computing system as a noncooperative stochastic game. We propose an approximation-based multiagent Markov decision process without the global system state observations, under which a multiagent proximal policy optimization (PPO) algorithm is derived to solve the corresponding Nash equilibrium. When expanding to a nonstationary heterogeneous edge computing system, the obtained algorithm suffers from the slow convergence due to constrained adaptability. Accordingly, we explore meta-learning and propose a multiagent meta-PPO algorithm, which rapidly adapts the control policy learning to the nonstationarity. Numerical experiments demonstrate performance gains from our proposed algorithms.