Visible and Infrared Image Fusion for Object Detection: A Survey

Thanks to the advancements of Deep Learning (DL) Algorithms. DL-based object detection models has witnessed remarkable success in the past few years. By leveraging deep convolutional neural networks (CNNs) and other deep learning models, rich feature representations can be effectively extracted from visible (RGB) images for object detection. However, for challenging scenarios such as low-light conditions and haze, the performance of object detection using visible images is often not satisfactory. On the other hand, thermal camera, which is unaffected by lighting conditions, can penetrate through low-light and hazy environments to capture object images. However, infrared images lack edge and texture information of objects. Recognizing the complementary nature of visible and infrared images, researchers have explored the fusion of visible and infrared images for object detection, yielding promising research outcomes. This paper provides an analysis of the current research status of visible and infrared image fusion for object detection. Firstly, the fusion models are categorized into three kinds: pixel-level fusion, feature-level fusion, and decision-level fusion. Several models within each category are discussed. Furthermore, this paper summarizes five datasets that can be utilized for training RGB-Infrared object detection models and compares the experimental results of selected models on the KAIST dataset. Lastly, the paper concludes with a summary of existing challenges in the field and offers some reflections on future directions.