SIGFusion: Semantic Information-Guided Infrared and Visible Image Fusion

In the field of image fusion, the fusion of infrared and visible images emerges as a crucial work, retaining the inherent characteristics of each source image for the creation of high-quality fusion images. However, existing fusion algorithms primarily prioritize the visual performance and statistical metrics, often neglecting the requisites of high-level vision tasks. To bridge this gap, this article proposes SIGFusion, an end-to-end method for infrared and visible image fusion. SIGFusion not only considers the visual performance and statistical metrics of the fused image but also focuses on meeting the demands of subsequent high-level vision tasks, thereby enhancing performance in fusion and high-level vision tasks. Concretely, we define significant semantic information as salient targets presenting in both infrared and visible images. By utilizing targets, our fusion network is guided to enhance these salient targets. In contrast to the isolated employ of either infrared or visible images, our target-based method comprehensively utilizes the semantic information within the image. In addition, the utilization of target-based self-attention loss enhances the extraction of vital information from the source image. It is worth noting that masks are only necessary during the training stage and are automatically generated through detection algorithms. We conduct experiments in both image fusion and high-level vision tasks. Experimental results demonstrate the effectiveness of SIGFusion in both infrared and visible image fusion tasks, as well as in subsequent high-level vision tasks. The source code is available at: https://github.com/poison-pig/SIGFusion.