A method for anomaly detection of track bed based on multimodal feature fusion

Anomaly detection of track bed is critical to the safety of trains. One problem that still exists is that when using 2D images for detection, the appearance of some anomalies is similar to the surrounding environment, resulting in false negatives. To solve this problem, we choose to use both 2D images and depth maps for anomaly detection, and propose a method for anomaly detection of track bed based on multimodal feature fusion. In our model, we use a student-teacher network with an asymmetric structure as the main network for anomaly detection. Furthermore, we innovatively introduce the interactive feature extraction module and the feature fusion module to efficiently fuse the features of different modalities. We evaluated the performance of our method on MVTec 3D-AD and a dedicated track bed foreign object dataset. The experiment results demonstrate that our method significantly outperforms other anomaly detection methods on MVTec 3D-AD. Furthermore, our method performed outstandingly on the track bed foreign object dataset. In terms of recall, it reached 96.32%, meeting the requirements of anomaly detection for track bed and validating its practicality.