Collaborative deep semi-supervised learning with knowledge distillation for surface defect classification

Defect inspection plays a vital role in ensuring high-quality production in industrial automation. While supervised approaches have been successful, they rely on costly labeled data. To address this limitation, semi-supervised methods have gained popularity, utilizing both labeled and unlabeled data for training. This research addresses the challenge of noisy semi-supervised training caused by incorrect pseudo-labels in Convolutional Neural Network based models. To enhance the accuracy and reliability of pseudo-label selection, a novel collaborative learning strategy with knowledge distillation for defect classification is proposed. The proposed approach involves training a set of teacher networks collaboratively, allowing them to collectively determine the pseudo-labels for each unlabeled image and improving the quality of pseudo-labeling. Subsequently, each teacher network is trained using these pseudo-labeled data. Finally, the acquired collaborative knowledge is transferred to a single student network, reducing model complexity, memory requirements, and enabling faster inference during deployment. The proposed approach demonstrates competitive performance on three publicly available defect classification datasets: NEU steel surfaces, SLS laser powder beds, and Surface Textures, achieving results comparable to the state-of-the-art. Notably, remarkable accuracy is achieved even with limited labeled data during training. For instance, on the SLS dataset, the proposed approach achieves 97% accuracy, which is comparable to the state-of-the-art's 98% accuracy when using 100% of labeled data. Remarkably, the proposed approach accomplishes this level of accuracy using only 3% of the labeled training data, showcasing its efficiency and effectiveness in leveraging limited labeled data to achieve impressive results. Source code is available at https://github.com/M-Siyamalan/CDSSLwithKD/.