A Novel Incremental Defect Detection Method via Elastic Heterogeneous Distillation Network

Abstract

In industrial processes, the defect data is continuously accumulated over time, and new classes of defects can arise at any time. When a well-trained model is adapted to new classes, its performance in old classes will sharply decline. To solve the above issue, a novel incremental defect detection method named elastic heterogeneous distillation network (EHD-Net) is proposed. First, an elastic knowledge transfer method is proposed to selectively transfer the core features of the old classes, so that the old knowledge is retained and new knowledge is effectively learned at the feature level. Second, warping heterogeneous distillation learning is proposed for the first time. In warping heterogeneous distillation learning, a gradient warping layer is proposed to balance the optimization direction of gradients for both old and new classes, and the proposed heterogeneous distillation learning strategy can clarify the association and difference between new and old classes, so that the rapid learning of new knowledge and comprehensive transfer of old knowledge are guaranteed at the decision level. Finally, a well-trained evolutionary model is employed to achieve the detection of both old and new classes. The proposed method can effectively overcome the catastrophic forgetting of old classes and guarantee the independent learning of new classes. Experimental results on two industrial datasets show that EHD-Net outperforms existing advanced methods. Note to Practitioners—The motivation for this paper is an important and highly practical issue called class-incremental defect detection. Specifically, new classes of defects always appear in industrial scenarios over time and when a well-trained detection model adapts to the new classes, its performance in the old classes decreases dramatically. To solve the above issues, an elastic heterogeneous distillation network (EHD-Net) via elastic knowledge transfer and warping heterogeneous distillation learning is proposed. The proposed method not only effectively suppresses the catastrophic forgetting of old classes while realizing the fast learning of new classes, so that the model can simultaneously achieve the high-accuracy detection of both the new and old classes without the participation of old class samples. The validity of the proposed method is verified under two industrial datasets, which fully guarantees its practical application value.

Topics & Concepts