DeepGuard: Efficient Anomaly Detection in SDN With Fine-Grained Traffic Flow Monitoring
Trung V. Phan, Tri Gia Nguyen, Nhu–Ngoc Dao, Trương Thu Hương, Nguyen Huu Thanh, Thomas Bauschert
Abstract
Software-Defined Networking (SDN) leverages the implementation of reliable, flexible and efficient network security mechanisms which make use of novel techniques such as artificial intelligence (AI) and machine learning (ML). In particular, these techniques - together with SDN - are the key enablers for the design of anomaly detection methods which are based on efficient traffic flow monitoring. In this paper, we tackle this problem by proposing an efficient anomaly detection framework, denoted as DeepGuard, which improves the detection performance of cyberattacks in SDN based networks by adopting a fine-grained traffic flow monitoring mechanism. Specifically, the proposed framework utilizes a deep reinforcement learning technique, i.e., Double Deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -Network (DDQN), to learn traffic flow matching strategies maximizing the traffic flow granularity while proactively protecting the SDN data plane from being overloaded. Afterwards, by implementing the learned optimal traffic flow matching control policy, the most beneficial traffic information for anomaly detection is acquired at runtime—thereby improving the cyberattack detection performance. The performance of the proposed framework is validated by extensive experiments, and the results show that DeepGuard yields significant performance improvements compared to existing traffic flow matching mechanisms regarding the level of traffic flow granularity. In the case of distributed denial-of-service (DDoS) attacks, DeepGuard achieves a remarkable attack detection performance while effectively preventing forwarding performance degradation in the SDN data plane.