Anteater: Advanced Persistent Threat Detection With Program Network Traffic Behavior
Yangzong Zhang, Wenjian Liu, Kaiian Kuok, Ngai Cheong
Abstract
Recent stealth attacks cleverly disguise malicious activities, masquerading as ordinary connections to popular online services through seemingly innocuous applications. These methods often evade detection by traditional network monitoring or signature-based techniques, as attackers frequently hide Command and Control (C&C) servers within well-known cloud service providers, making the traffic anomalies appear normal. In this paper, we introduce an application-level monitoring system, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Anteater</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Anteater</i> constructs a detailed profile for each legitimate software’s network traffic behavior, outlining the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">expected</i> traffic patterns. By scrutinizing a program’s network traffic configuration, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Anteater</i> efficiently pinpoints and intercepts the IP addresses associated with abnormal program access. Implemented in a real-world enterprise environment, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Anteater</i> was tested on a dataset containing over 400 million real-world network traffic sessions. The evaluation results demonstrate that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Anteater</i> achieves a high detection rate for malware injections, boasting a true positive rate of 94.5% and a false positive rate of less than 0.1%.