Cloud-Edge–End Collaborative Task Offloading in Vehicular Edge Networks: A Multilayer Deep Reinforcement Learning Approach
Jiaqi Wu, Ming Tang, Changkun Jiang, Lin Gao, Bin Cao
Abstract
Mobile-edge computing (MEC) is a promising computing scheme to support computation-intensive AI applications in vehicular networks, by enabling vehicles to offload computation tasks to edge computing servers deployed on road side units (RSUs) that approximate to them. In this work, we consider an MEC-enabled vehicular edge network (VEN), where each vehicle can offload tasks to edge/cloud computing servers via vehicle-to-infrastructure (V2I) links or to other end-vehicles via vehicle-to-vehicle (V2V) links. In such a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">cloud-edge–end</i> collaborative offloading scenario, we focus on the joint task offloading, scheduling, and resource allocation problem for vehicles, which is challenging due to the online and asynchronous decision-making requirement for each task. To solve the problem, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Multilayer deep reinforcement learning</i> (DRL)-based approach, where each vehicle constructs and trains three modules to make different layers’ decisions: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Offloading Module</i> (first layer), determining whether to offload each task, by using the dueling and double deep <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i>-network (D3QN) framework; 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Scheduling Module</i> (second layer), determining where and how to offload each task in the offloading queues, together with the transmission power, by using the parameterized deep <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i>-network (PDQN) framework; and 3) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Computing Module</i> (third layer), determining how much computing resource to be allocated for each task in the computation queues, by using classic optimization techniques. We provide the detailed algorithm design and perform extensive simulations to evaluate its performance. Simulation results show that our proposed algorithm outperforms the existing algorithms in the literature, and can reduce the average cost by 25.86%–72.51% and increase the average satisfaction rate by 3.48%–90.53%.