Multi-Agent Reinforcement Learning for Task Offloading in Crowd-Edge Computing
Su Yao, Mu Wang, Ju Ren, Tianyu Xia, Weiqiang Wang, Ke Xu, Mingwei Xu, Hongke Zhang
Abstract
The Crowd-edge (CE) computing paradigm facilitates the utilization of the computational resources through simultaneously relying the edge computing and the collaboration among various mobile devices (MDs). Most existing works, focusing on offloading tasks from device to edge servers by centralized solutions, are unable to distribute tasks to massive MDs in CE. Meanwhile, designing a decentralized task offloading solution enabling task subscribers to individually make offloading decisions can be challenging given the randomness of crowd resource provisioning and limited knowledge of global status variations. In this paper, we propose a decentralized crowd-edge task offloading solution that enables users to optimally offload tasks to the CE in a distributed manner. Specifically, we formulate the corresponding problem as a stochastic optimization with partially observable status. By observing network and process delays at the crowd side, we further reform the optimization forms and provide a novel approximation policy, enabling users to optimize their offloading strategy based on local observations without interaction with each other. We then solve this task offloading problem by developing a Mixed Multi-Agent Proxy Policy Optimization algorithm (mixed MAPPO). Extensive testing, including numerical and system-level simulations, was conducted to validate the performance of the proposed algorithm in terms of task delay (including the processing delay and transmission delay), load rate, and resource utilization.