Anomaly Detection and Localization via Reverse Distillation With Latent Anomaly Suppression
Gang Wang, Yisheng Zou, Songlin He, Yakun Wang, Rui-hong Dai
Abstract
Image anomaly detection and localization have received widespread interest in the community, and knowledge distillation (KD) has been widely explored. Recently, the reverse distillation (RD) paradigm has successfully mitigated the homogenization of anomaly representations in traditional KD arising from identical or similar teacher-student (T-S) architecture. However, in RD, the lack of an effective means to prevent anomalous patterns in the teacher encoder from being leaked into the student decoder undermines potential modeling discrepancies between the T-S model in anomaly representations. To settle this problem, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">REverse distillation with latent Anomaly SuppressiON</i> (REASON) method, preventing the student decoder from receiving anomalous patterns by extra means of anomaly filtering during the inference phase, and thus, the student model can only restore representations of anomaly-free images. Specifically, we construct a Siamese teacher encoder architecture, with one branch extracting features from anomaly-free samples and the other synthesizing anomaly features with spatial noise injection from the latent feature level. Next, we design a latent anomaly suppression module to recover normal features from perturbed anomalous features. In this sense, the follow-up student decoder will receive input without abnormal patterns. Thus, representations of the anomaly-free images can be described well, while those of the anomalous images can be well-differentiated between the T-S model. Furthermore, to enhance the model’s anomaly detection and localization capabilities, we propose multi-granularity KD loss to optimize the student decoder to focus on context and local details. Extensive experiments are performed on three benchmark datasets, i.e., MVTec AD, AeBAD, and OCT2017, and the results show the effectiveness and robustness of our proposed approach, which achieves superiority over the current state-of-the-art methods.