DeepCQF: Making CQF Scheduling More Intelligent and Practicable
Zongrong Cheng, Dong Yang, Weiting Zhang, Jie Ren, Hongchao Wang, Hongke Zhang
Abstract
Recently, Cyclic Queuing and Forwarding (CQF) based shaper has been proposed by IEEE 802.1Qch to satisfy the demands for deterministic industrial services. However, CQF only defines the queuing model and flow features, the investigations of scheduling for Time-Triggered flows are still at the early stage. How to make the scheduling more efficient and intelligent is still an open issue to be solved. In this paper, we propose a Deep Reinforcement Learning (DRL) assisted two-stage scheduling algorithm, namely DeepCQF, to solve the temporal scheduling problem based on CQF model, which can be adopted in the off-line and on-line scenarios. Due to the high schedulability and low time cost requirements in the off-line and on-line scenarios, respectively, DeepCQF uses the DRL algorithm to take immediate response for the coming flows when scheduling online, while in the off-line scenario, DeepCQF combines the DRL and heuristic algorithms together as a two-stage mechanism to further improve the schedulability. We compare the DeepCQF with other state-of-the-art algorithms for CQF. Experiment results show that DeepCQF can increase the number of accommodating flows by an average of 2 times than Greedy algorithm and the convergence speed by 3 times at most than Double Deep Q-network algorithm.