人工智能
计算机科学
残余物
计算机视觉
图像融合
模式识别(心理学)
量化(信号处理)
融合
相互信息
安全性令牌
图像处理
图像配准
图像(数学)
变压器
传感器融合
推论
翻译(生物学)
自回归模型
像素
图像传感器
作者
Honglin Wu,Junjie Huang,Huibin Tan,Wanrong Huang,Yuhua Tang,Xueqiong Li
出处
期刊:
[Institute of Electrical and Electronics Engineers]
日期:2025-06-30
卷期号:: 1-6
被引量:1
标识
DOI:10.1109/icme59968.2025.11208898
摘要
Infrared-visible image fusion (IVF) is an essential task in multimodal image processing that integrates infrared and visible modalities to enhance the overall image information content. However, existing methods often suffer from limited precision and efficiency. Furthermore, they fail to address practical requirements such as multi-resolution fusion and mutual translation. In this paper, we propose the Multi-Scale Residual Quantized Infrared-Visible Image Fusion (M-RQIVF) framework to efficiently generate high-quality fusion images. M-RQIVF trains multi-scale residual quantized infrared and visible autoencoders that convert images into multi-scale discrete token maps. This approach approximates the residuals from the features on a scale-by-scale basis, allowing for coarse-to-fine fused image generation that aligns well with human visual perception. Furthermore, by leveraging these discrete token maps, we train Visual Auto-Regressive (VAR) transformers using next-scale prediction. The VAR transformer ensures that features of corresponding sizes can be generated, even when the input infrared and visible images have different resolutions, facilitating fine-grained fusion. Additionally, the autoregressive structure enables image translation to be treated as a conditional generation task, thereby enabling mutual translation between infrared and visible images. Extensive experiments demonstrate that M-RQIVF outperforms the SOTAs while maintaining a much faster inference speed.
科研通智能强力驱动
Strongly Powered by AbleSci AI