RMDDQN-Learning: Computation Offloading Algorithm Based on Dynamic Adaptive Multi -Objective Reinforcement Learning in Internet of Vehicles
Xiangjun Zhang, Weiguo Wu, Zhihe Zhao, Jinyu Wang, Song Liu
Abstract
As a promising computing paradigm driven by 5 G, mobile edge computing (MEC) empowers smart vehicles to offload computation-intensive tasks to edge devices in the Internet of Vehicles (IoV), thereby providing a plethora of exciting applications (e.g. on-board AR/VR, autonomous driving, etc.) with the unique quality of service (QoS) guarantees. However, a key challenge of MEC is how to keep the delay and energy consumption to a minimum in the computation offloading process, while ensuring the privacy security of offloaded data and the load balancing of edge resources. Namely, how to simultaneously optimize multiple indicators that affect computation offloading, which creates a challenging multi-objective optimization (MOO) problem. Aiming at the MOO problem, we propose a novel multi-objective reinforcement learning (MORL) algorithm based on double deep Q-network (DDQN). Each DDQN agent obtains rewards on different objectives according to different reward functions, and dynamically approximates the optimal offloading decision on multiple objectives. To conquer the trade-off problem among multiple conflicting objectives, we propose a weight-learning network based on radial basis function (RBF) networks, which dynamically adjusts the weights by learning the value changes among the objectives. Interestingly, we discovered encouraging potential of MORL for solving computation offloading problems in IoV, and numerical results show that the proposed algorithm outperforms traditional reinforcement learning methods by 30.24% in terms of overall energy consumption.