Effective IR-RGB image registration and fusion method for railway scene

Railway perimeter protection is crucial for ensuring railway operation safety. Video surveillance technology is widely used in it, but visible light cameras often suffer from poor imaging quality under night time or adverse weather conditions, which affects the effectiveness of protection. The fusion of RGB and infrared (IR) images can improve image quality, thereby enhancing subsequent railway intrusion detection. To fully utilize the characteristics of railway scenes to achieve accurate image registration, a coarse-to-fine image registration method based on line features is designed. For the registered image pairs, to ensure the fused image contains both the texture and contour details of RGB image and the temperature information of IR image, an image fusion method based on Non-Subsampled Contourlet Transform(NSCT)is proposed, specifically, a new saliency measurement (NSM) to measure the multi information of image, including structure, sharpness, and brightness is for low-frequency sub-band fusion, fourth-order correlation coefficients (FOCC)matching is for high-frequency sub-band fusion. Experiments demonstrate that the proposed method performs better in several evaluation metrics compared with other four traditional algorithms, displaying richer information and high resolution.