Cybertwin-Driven DRL-Based Adaptive Transmission Scheduling for Software Defined Vehicular Networks
Wei Quan, Mingyuan Liu, Nan Cheng, Xue Zhang, Deyun Gao, Hongke Zhang
Abstract
Efficient transmission control is a challenging issue in vehicular networks due to the highly dynamic and unpredictable link status. In this paper, we propose a cybertwin-driven learning-based transmission scheduling mechanism for software-defined vehicular networks, which can adaptively select/adjust transmission control methods, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> loss-based, delay-based and hybrid ones, to suit to the time-varying network environment. In particular, we first analyze the dynamic network characteristics of three realistic vehicular network scenarios in terms of network throughput, round-trip time (RTT) and RTT jitter. Furthermore, we propose a novel transmission scheduling model and formulate the SDVN transmission scheduling issue as a linear programming problem. To obtain the optimized scheduling policies and guarantee the effectiveness of transmission control, we further propose a Cybertwin-driven and Deep Reinforcement Learning based transmission control solution (TcpCDRL). Specifically, TcpCDRL is featured with: (i) using deep reinforcement learning (DRL) to adaptively adjust transmission control policy, (ii) using cybertwin-driven transmission controlling to improve the policy-making effectiveness and timeliness. Simulation results show that the proposed TcpCDRL approach outperforms the single well-known transmission control approach ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., TcpWestwood, TcpBic, TcpVeno and TcpVegas) in terms of network throughput and RTT.