Reverse Distillation for Continuous Anomaly Detection

Unsupervised anomaly detection and localization methods only use anomaly-free images to train the network. Ultimately, the network should be able to detect whether the input image contains anomalies and to locate the anomalous areas. There has been a lot of related research. Most of the existing research still stays in the stage of training separate models for each category. However, in industrial applications, such task setting is costly and time-consuming. Our work is to study anomaly detection methods in the continual learning setting. In this work, we use the reverse teacher-student (T-S) distillation model as the backbone network to detect anomalies in samples. To make the model able to learn in a sequence of tasks, we perform pooling distillation on the feature tensors from the student model and the embedding representations. Then, the model in the new task can retain the knowledge of the model in the previous tasks. To verify the performance of the proposed method in practical application scenarios, we also introduce a printed circuit board (PCB) defect detection dataset for continual learning tasks. This dataset divides PCB samples into multiple anomaly detection tasks based on different capturing locations, which can be used to perform validation experiments for anomaly detection algorithms based on continual learning methods. The experimental results on the MVTec AD dataset and the PCB dataset show that the detection performance of the proposed method is superior to the existing state-of-the-art (SOTA) T-S distillation anomaly detection methods in the continual learning setting. The average pixel-level AUROC (P-AUROC) reaches 0.847 on the MVTec AD dataset.