PENet-KD: Progressive Enhancement Network via Knowledge Distillation for Rail Surface Defect Detection

As an essential transportation system in modern society, the significance of railway track safety cannot be overlooked. In recent years, computer vision systems and deep learning have been increasingly applied to unserved track defect detection. Although several algorithms have been proposed to address safety concerns, there is a need to enhance their efficiency and accuracy. This study introduces an efficient progressive enhancement network via knowledge distillation (PENet-KD) for detecting defects on the rail surface. In PENet-KD, we utilize knowledge distillation to transfer the expertise of the teacher network to the student network, resulting in a lightweight model with high speed and accuracy. Additionally, two modules were developed to gradually refine features. Initially, cross-modal information is dynamically fused using a regenerative high-level attention module based on a graphical convolutional network, which corrects the features derived from the encoder. Subsequently, in the decoding stage, significant semantic guidance information is obtained by applying three-dimensional attentional optimization to the highest layer features, thereby guiding the progressive distillation module to produce precise outcomes. Extensive experiments conducted on an industrial RGB-D NEU RSDDS-AUG benchmark dataset demonstrate that the proposed PENet-KD outperforms the existing state-of-the-art methods, thus showcasing its generality and effectiveness. Notably, on the RSDDS-AUG dataset, PENet-KD achieved a maximum E-measure gain of 1.4% and a S-measure gain of 1.2% compared to the best current method. The code and models utilized in this research are publicly available at https://github.com/Wang-5ying/PENet-KD.