鉴别器
人工智能
计算机科学
图像融合
模式识别(心理学)
图像(数学)
保险丝(电气)
计算机视觉
发电机(电路理论)
图像分辨率
物理
探测器
量子力学
电信
功率(物理)
作者
Jiayi Ma,Han Xu,Junjun Jiang,Xiaoguang Mei,Xiaoping Zhang
出处
期刊:IEEE transactions on image processing
[Institute of Electrical and Electronics Engineers]
日期:2020-01-01
卷期号:29: 4980-4995
被引量:511
标识
DOI:10.1109/tip.2020.2977573
摘要
In this paper, we proposed a new end-to-end model, termed as dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Our method establishes an adversarial game between a generator and two discriminators. The generator aims to generate a real-like fused image based on a specifically designed content loss to fool the two discriminators, while the two discriminators aim to distinguish the structure differences between the fused image and two source images, respectively, in addition to the content loss. Consequently, the fused image is forced to simultaneously keep the thermal radiation in the infrared image and the texture details in the visible image. Moreover, to fuse source images of different resolutions, e.g., a low-resolution infrared image and a high-resolution visible image, our DDcGAN constrains the downsampled fused image to have similar property with the infrared image. This can avoid causing thermal radiation information blurring or visible texture detail loss, which typically happens in traditional methods. In addition, we also apply our DDcGAN to fusing multi-modality medical images of different resolutions, e.g., a low-resolution positron emission tomography image and a high-resolution magnetic resonance image. The qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our DDcGAN over the state-of-the-art, in terms of both visual effect and quantitative metrics.
科研通智能强力驱动
Strongly Powered by AbleSci AI