MADRL-Based URLLC-Aware Task Offloading for Air-Ground Vehicular Cooperative Computing Network
Peng Qin, Wang Yi-fei, Ziyuan Cai, Jiayan Liu, Jinghan Li, Xiongwen Zhao
Abstract
With the rapid development of 5G and Internet of Vehicles (IoV) technologies, vehicles have evolved from mere transportation devices to mobile living spaces for humans. Thus, the complexity of data processing is growing along with the increasing of vehicle applications, which makes relying solely on on-board processing capabilities insufficient. Traditional Roadside Units (RSUs)-based edge computing has limitations such as high deployment cost and finite coverage. To address those issues, we construct an Air-Ground Vehicular Cooperative Computing Network (AVC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{\textbf{2}}$</tex-math> </inline-formula> N) that introduces Cooperative Vehicles (CVs) to reduce deployment cost while incorporating Unmanned Aerial Vehicles (UAVs) to expand communication coverage. We aim to balance offloading efficiency and environmental sustainability by proposing a system cost minimization problem with weighted sum of delay and energy consumption. However, it faces new challenges such as the lack of Global State Information (GSI), and the Ultra-Reliable Low-Latency Communications (URLLC) queue delay constraints, rendering traditional methods inadequate. To address the coupling between immediate decision and long-term queuing constraints, we employ Lyapunov optimization to partition the initial problem into two distinct sub-problems. The first focuses on optimizing the system transmission cost, which is tackled using a collaborative Deep Reinforcement Learning (DRL) framework. Specifically, we design an algorithm based on Multi Agent Deep Deterministic Policy Gradient (MADDPG), which effectively addresses GSI uncertainty and ensures URLLC awareness. The second sub-problem addresses server-side computing resource optimization, and we propose a greedy algorithm to tackle it. Experimental results showcase the effectiveness of our approach, demonstrating notable achievement in terms of learning convergence speed, overall system cost, queue delay, and queue backlog.