计算机视觉
透视图(图形)
人工智能
图像融合
计算机科学
图像处理
融合
可视化
传感器融合
红外线的
图像(数学)
图像配准
迭代重建
图像分割
目标检测
模式识别(心理学)
图像传感器
特征提取
光学成像
遥感
图像复原
高光谱成像
作者
Hengshuai Cui,Weidong Zhao,Haipeng Wang,Jun Yan,Hao Liu
标识
DOI:10.1109/tip.2026.3737049
摘要
Existing fusion methods integrate complementary information from source images to generate a mixed-modality fused image, causing visually ambiguous representations that degrade downstream task performance (e.g., segmentation and detection). Therefore, we rethink infrared and visible image fusion from a modality-invariant perspective that reconstructs two real-modality fusion images (RealFusion), where the two challenges of cross-modality feature gap and modality inconsistency are addressed. Firstly, we introduce a scene prompt generation that leverages a semantic segmentation task to extract holistic scene information from source images, thereby bridging the gap between visible and infrared features. Secondly, we design a real-modality fusion image reconstruction that constructs the modality-guided vectors between the embedding space and each modality-specific space as the projection directions, thus projecting the fused features to their respective modality spaces. Thus, the fusion network is forced to learn a consistent representation from real-modality images, which enables the fusion results to keep the modality invariant. Additionally, we introduce a differential interaction fusion module that explicitly formulates the infrared and visible features as common and differential components, and then dynamically enhances the differential information that obtains a complementary fusion representation. Extensive experiments on four datasets demonstrate that RealFusion outperforms the state-of-the-art methods on both the fusion task and the downstream segmentation task. The code will be released at: https://github.com/HengshuaiCui/RealFusion.
科研通智能强力驱动
Strongly Powered by AbleSci AI