A hierarchical feature-logit-based knowledge distillation scheme for internal defect detection of magnetic tiles

Magnetic tiles are the key components of various electrical and mechanical systems in modern industry, and detecting their internal defects holds immense significance in maintaining system performance and ensuring operational safety. Recently, deep learning has emerged as a leading approach in pattern recognition due to its strong capability of extracting latent information. In practical scenarios, there is a growing demand for embedding deep learning algorithms in edge devices to enable real-time decision-making and reduce data communication costs. However, a powerful deep learning algorithm with high complexity is impractical for deployment on edge devices with limited memory capacity and computational power. To overcome this issue, we propose a novel knowledge distillation method, entitled hierarchical feature-logit-based knowledge distillation, to compress deep neural networks for internal defect detection of magnetic tiles. Specifically, it comprises a one-to-all feature matching for disparate feature knowledge distillation, a logit separation for relevant and irrelevant logit knowledge distillation, and a parameter value prediction network for seamlessly fusing feature and logit knowledge distillation. Besides, an ingenious hierarchical distillation mechanism is designed to address the capacity gap issue between the teacher and the student. The extensive experimental results show the effectiveness of our proposed model. The code is available at https://github.com/Clarkxielf/A-hierarchical-feature-logit-based-knowledge-distillation-scheme-for-internal-defect-detection.