Joint Differential Game and Double Deep Q-Networks for Suppressing Malware Spread in Industrial Internet of Things

Abstract

Industrial Internet of Things (IIoT), which has the capability of perception, monitoring, communication and decision–making, has already exposed more security problems that are easy to be invaded by malware because of many simple edge devices that help smart factories, smart cities and smart homes. In this paper, a two–layer malware spread–patch model <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IIPV</i> is proposed based on a hybrid patches distribution method according to the simple edge equipments and limited central computer resources of IIoT. The spread process of malware in IIoT was deeply analyzed using differential game and a differential game model was established. Then optimization theory was further used to solve the optimization problem extracted by introducing subjective effort parameters to obtain the optimal control strategies of devices for malware and patches. In addition, we combined the deep reinforcement learning algorithm into the model <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IIPV</i> to design a new algorithm <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDQN</i> – <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PV</i> suitable for suppressing the spread of malware in IIoT during the experiments. Finally, the effectiveness of model <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IIPV</i> and algorithm <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDQN</i> – <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PV</i> are verified by numerous comparative experiments.

Topics & Concepts