The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection
Lars Heckler-Kram, Jan-Hendrik Neudeck, Ulla Scheler, Rebecca König, Carsten Steger
Abstract
In recent years, performance on existing anomaly detection benchmarks like MVTecAD and VisA has started to saturate in terms of segmentation AU-PRO, with state-of-the-art models often competing in the range of less than one percentage point. This lack of discriminatory power prevents a meaningful comparison of models and thus hinders progress of the field, especially when considering the inherent stochastic nature of machine learning results. We present the MVTecAD2 dataset, a collection of advanced anomaly detection scenarios with more than 8000 high-resolution images from eight object categories. It comprises challenging and highly relevant industrial inspection use cases that have not been considered in previous datasets, including transparent and overlapping objects, dark-field and backlight illumination, objects with high variance in the normal data, and extremely small defects. We provide comprehensive evaluations of state-of-the-art methods and show that their performance remains below 60% average AU-PRO. Additionally, our dataset provides test scenarios with lighting condition changes to assess the robustness of methods under real-world distribution shifts. We host a publicly accessible evaluation server that holds the pixel-precise ground truth of the test set ( https://benchmark.mvtec.com ). All image data is available at https://www.mvtec.com/company/research/datasets/mvtec-ad-2 .