图像配准
人工智能
计算机视觉
计算机科学
图像融合
融合
图像处理
传感器融合
图像(数学)
图像分割
特征提取
模式识别(心理学)
可视化
多模态
稳健性(进化)
像素
目标检测
医学影像学
作者
Xuheng Liu,Rencan Nie,Jinde Cao,Guangxu Xie,Yi Zhong
标识
DOI:10.1109/tcsvt.2026.3723110
摘要
Joint multimodal registration and fusion frameworks are often limited by architectural limitations, particularly the lack of mutual interaction between tasks and the blurring induced during the resampling of registered images. This loss of clarity in the registration stage directly degrades the quality of the subsequent fusion results. To address these issues, we propose CL-RFNet, an end-to-end closed-loop framework. To reduce resampling blur, we extract a source high-frequency map from the original moving image and convert it into semantic prompts through HFInjector to guide fusion reconstruction. Furthermore, we introduce a contrastive learning paradigm to establish a dynamic link between the two tasks. The current fused result is used as the anchor, while fused results from aligned and misaligned image pairs serve as positive and negative samples, respectively. Registration errors introduce artifacts into fused images and increase the contrastive loss. This loss converts fusion artifacts caused by residual misalignment into an optimization signal for deformation-field refinement. In the registration stage, we leverage the geometric correlation of semantic points for correspondence matching and refine features through a progressive hierarchical optimization strategy. During the fusion stage, we employ a modality balancer to estimate informational contributions. The high-frequency semantic prompts further guide the decoder for detail-aware reconstruction. Experiments on the RoadScene, M3FD, and TNO datasets demonstrate that our method achieves competitive performance in both quantitative metrics and visual quality across joint and standalone tasks.
科研通智能强力驱动
Strongly Powered by AbleSci AI