人工智能
图像融合
计算机科学
像素
模式识别(心理学)
分类器(UML)
计算机视觉
失真(音乐)
融合
融合规则
图像(数学)
上下文图像分类
特征提取
特征(语言学)
带宽(计算)
计算机网络
哲学
语言学
放大器
作者
Han Xu,Hao Zhang,Jiayi Ma
出处
期刊:IEEE transactions on computational imaging
日期:2021-01-01
卷期号:7: 824-836
被引量:188
标识
DOI:10.1109/tci.2021.3100986
摘要
Existing image fusion methods always use hand-crafted fusion rules due to the uninterpretability of deep feature maps, which restrict the performance of networks and result in distortion. To address these limitations, this paper for the first time realizes the interpretable importance evaluation of feature maps in a deep learning manner. This importance-oriented fusion rule helps preserve valuable feature maps and thus reduce distortion. In particular, we propose a pixel-wise classification saliency-based fusion rule. First, we employ a classifier to classify two types of source images which capture the differences and uniqueness between two classes. Then, the importance of each pixel is quantified as its contribution to the classification result. The importance is shown in the form of classification saliency maps. Finally, the feature maps are fused according to the saliency maps to generate fusion results. Moreover, because there is no need of manually deciding the characteristics to be retained, it is an unsupervised method with less human participation. Both qualitative and quantitative experiments demonstrate the superiority of our method over the state-of-the-art fusion methods even if using a simple network.
科研通智能强力驱动
Strongly Powered by AbleSci AI