A real-time railway fastener inspection method using the lightweight depth estimation network

Fasteners are critical components of railways that maintain the rail tracks in a fixed position. Their failure can lead to serious accidents such as train derailments, so their condition needs to be inspected periodically. Conventional image-based inspection methods fail to take full advantage of the structural features of fasteners, making them less robust in real-world environments. This paper presents a new approach for real-time fastener inspection by (1) extracting fastener regions using the YOLOv3-tiny network (2) proposing and pruning a lightweight and encoder–decoder network architecture for inferring depth information from a single RGB image of fasteners (3) fusing the RGB-D features for inspection. Compared with the image-based SVM, the F 1 of RGB-D fusion-based SVM increases from 94.34% to 95.83%, illustrating the improvement of additional depth information for fastener defect inspection. The inspection system runs at 11.9 FPS, which enables real-time inspection of railway fasteners. • A novel depth estimation network named MobileNetV2-BlinearUp is proposed for the fastener inspection. • The RGB and estimated depth data are fused together for classification with the SVM method. • It has the least parameters and gives comparable and visually smooth results. • It improves the F 1 from 94.34% to 95.83% compared to RGB image-based SVM. • The effectiveness and efficiency of the inspection system meet the requirements of on-line fastener inspection.