编码器
计算机科学
人工智能
融合
GSM演进的增强数据速率
图像融合
融合规则
计算机视觉
比例(比率)
模式识别(心理学)
图像(数学)
语言学
量子力学
操作系统
物理
哲学
作者
Xiangfeng Luo,Qianqian Wang,Zhancheng Zhang,Xiaojun Wu
标识
DOI:10.1016/j.patcog.2023.110192
摘要
Most existing deep learning-based infrared and visible image fusion methods always fail to consider the full-scale long-range correlation and the prior knowledge, resulting in the fused images with low-contrast salient objects and blurred edge details. To overcome these drawbacks, a full-scale hierarchical encoder-decoder network with cascading edge-prior for infrared and visible image fusion is proposed. First, a top-down encoder extracts the hierarchical representations from source image. Then, to inject edge priors into the network and capture the progressive semantic correlations, a triple fusion mechanism is proposed including edge image fusion based on maximum fusion strategy, single-scale shallow layer fusion and full-scale semantic layer fusion based on dual-attention fusion (DAF) strategy. The fused full-scale semantic features (F2SF) are obtained by capturing the long-range affinities of the full-scale. At the same time, a cascading edge-prior branch (CEPB) is designed to embed the fused edge knowledge into fused single-scale shallow features, jointly guiding the decoder to focus on abundant details layer-by-layer on the basis of F2SF, thus recovering the edge and texture details of the fused image well. Finally, a novel loss function consisting of SSIM, intensity and edge loss is constructed to further maintain the network with better edge representation and reconstruction capability. Compared with existing state-of-the-art fusion methods, the proposed method has better performance in terms of both visual evaluation and objective evaluation on public datasets. The source code is available at https://github.com/lxq-jnu/FSFusion.
科研通智能强力驱动
Strongly Powered by AbleSci AI