计算机科学
稳健性(进化)
人工智能
融合
图像融合
特征(语言学)
红外线的
编码器
计算机视觉
特征提取
模式识别(心理学)
保险丝(电气)
图像(数学)
工程类
化学
语言学
哲学
生物化学
物理
光学
电气工程
操作系统
基因
作者
Fan Zhao,Wenda Zhao,Libo Yao,Yu Liu
标识
DOI:10.1016/j.inffus.2021.06.002
摘要
Benefitting from the strong feature extraction capability of deep learning, infrared and visible image fusion has made a great progress. Since infrared and visible images are obtained by different sensors with different imaging mechanisms, there exists domain discrepancy, which becomes stumbling block for effective fusion. In this paper, we propose a novel self-supervised feature adaption framework for infrared and visible image fusion. We implement a self-supervised strategy that facilitates the backbone network to extract features with adaption while retaining the vital information by reconstructing the source images. Specifically, we preliminary adopt an encoder network to extract features with adaption. Then, two decoders with attention mechanism blocks are utilized to reconstruct the source images in a self-supervised way, forcing the adapted features to contain vital information of the source images. Further, considering the case that source images contain low-quality information, we design a novel infrared and visible image fusion and enhancement model, improving the fusion method’s robustness. Experiments are constructed to evaluate the proposed method qualitatively and quantitatively, which show that the proposed method achieves the state-of-art performance comparing with existing infrared and visible image fusion methods. Results are available at https://github.com/zhoafan/SFA-Fuse.
科研通智能强力驱动
Strongly Powered by AbleSci AI