ContexLog: Non-Parsing Log Anomaly Detection With All Information Preservation and Enhanced Contextual Representation
Ruizhi Xiao, Weilong Li, Jintian Lu, Shuyuan Jin
Abstract
Logs are widely used in software to trace the runtime states and critical events. Log-based anomaly detection is crucial for software maintenance and reliability assurance. Existing log-based anomaly detection methods are suffering from imperfections of log parsing, the neglect of the log individual context, and the discarding of non-character tokens. In this paper, we propose ContexLog, a non-parsing log-based anomaly detection method with all information preservation and enhanced log contextual representation, to detect diverse anomalies effectively. Log messages are first grouped as sequences with different windowing techniques. To capture all log features, ContexLog tokenizes each log sequence and preserves all information, including character and non-character tokens. It then represents the log sequential context and individual context simultaneously to construct input for a Transformer encoder-based classification model. Experimental evaluations on real-world datasets and synthetic datasets demonstrate ContexLog outperforms existing methods in achieving accurate anomaly detection results, handling unseen logs to avoid log parsing imperfections, and utilizing non-character tokens to detect diverse anomalies.