Research on multimodal techniques for arc detection in railway systems with limited data

The pantograph–catenary system is a critical component of railway vehicles, and its performance directly affects the quality of current collection. Accurately measuring the arcing rate is essential for monitoring the system’s condition and ensuring safe operation. However, traditional arc detection methods are prone to increased false detection rates and reduced measurement accuracy in complex railway environments due to the diversity of arc sizes and shapes, environmental interference, instability in current collection, and power fluctuations. While deep learning-based methods can effectively address environmental interference, obtaining sufficient labeled training data is challenging because arc events occur infrequently. Moreover, a large number of unlabeled images of pantograph–catenary contacts cannot be directly utilized due to the lack of annotations. To solve these issues, a novel arc detection method is proposed: a multimodal arc detection network based on denoising diffusion probabilistic models (DDPMs-MILNet). First, a DDPM is pretrained using a large set of unlabeled images to acquire advanced image features. This model serves as a feature extractor, and a hierarchical variation semantic decoder is fine-tuned, thereby improving performance under small-sample conditions and reducing dependence on extensive labeled datasets. Building on this, an audiovisual semantic decoder is designed to incorporate audio signals as semantic cues, providing additional modality information for visual features. This approach not only reduces the model’s reliance on visual information but also enables it to locate the visual target of the arc even when the object is not simultaneously seen and heard, further alleviating the challenges posed by limited sample sizes. Experimental results demonstrate that DDPM-MILNet achieves excellent detection performance with minimal data in complex railway environments, indicating significant application potential, particularly in the state monitoring and anomaly detection of railway systems.