Deep Reinforcement Learning-Based Cloud-Edge Collaborative Mobile Computation Offloading in Industrial Networks
Siguang Chen, Jiamin Chen, Yifeng Miao, Qian Wang, Chuanxin Zhao
Abstract
With the rapid development of mobile industrial applications and due to the limited coverage of static edge servers, traditional edge computing technology has great limitations in dynamic environmental applications. This paper proposes a deep reinforcement learning-based cloud-edge collaborative mobile computation offloading mechanism for satisfying the dynamic service requirements in industrial networks. Specifically, a three-layer network model of digital twins and a decentralized network of task resources are first constructed to handle the mobility of user terminals and the relevance of tasks. Then, based on the comprehensive consideration of mobility, associated tasks, computing resources and offloading decisions, an optimization problem is formulated to minimize the weighted sum of the execution delay and energy consumption of all tasks for all users. Additionally, a deep reinforcement learning-based cloud-edge collaborative mobile computation offloading (DRL-CCMCO) algorithm is proposed to solve this optimization problem. Based on the differences in each edge cloud, this algorithm sets the priority of the shared experience pool and selects the most effective experience samples to complete better learning and training. It also utilizes a distributed learning method to learn the probability of an approximate reward distribution and optimizes network parameters through cloud-edge collaboration to achieve faster optimal offloading decision. Finally, a large number of simulation results show that the proposed algorithm has the characteristics of fast convergence and high stability, and it can obtain the optimal offloading decision with the lowest total cost.