A Deep Reinforcement Learning Approach to Time Delay Differential Game Deception Resource Deployment
Weizhen He, Jinglei Tan, Ruixiang Wang, Zhiquan Liu, Xiangyang Luo, Hongchao Hu, Hengwei Zhang
Abstract
Current methods for deploying cyber deception do not consider the impact of time delays on the effectiveness of actions by both attackers and defenders, nor can they make real-time decisions on the deployment of deception assets in complex network environments. To address these issues, this paper proposes a deception resource deployment method based on deep reinforcement learning with time-delay differential game theory. First, we constructed the security evolution process of nodes in complex network environments by analyzing the threat models of attackers and defense models of defenders, presenting time-delay differential state equations for nodes with varying degrees. Furthermore, we introduced a cyber deception time-delay differential game model, quantifying the gains for both sides. We then designed a deep reinforcement learning algorithm, employing proximal policy optimization (PPO) to determine the optimal deception deployment strategy, based on the analysis of the network deception time-delay differential game model. Finally, the effectiveness of the proposed method in determining the optimal deception deployment strategy was validated through the construction of a scale-free complex network. Experimental results show that the proposed model could effectively discern the evolutionary processes of nodes with different degrees and the strategies of both attackers and defenders. Compared with other methods, the proposed method showed distinct advantages in stability and effectiveness. The results indicate that the proposed method can be effectively deployed in cyber deception.