亲爱的研友该休息了!由于当前在线用户较少,发布求助请尽量完整地填写文献信息,科研通机器人24小时在线,伴您度过漫漫科研夜!身体可是革命的本钱,早点休息,好梦!

Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity

计算机科学 人工智能 图像融合 计算机视觉 杠杆(统计) 特征(语言学) 融合 分割 图像(数学) 模式识别(心理学) 语言学 哲学
作者
Linfeng Tang,Hao Zhang,Han Xu,Jiayi Ma
出处
期刊:Information Fusion [Elsevier BV]
卷期号:99: 101870-101870 被引量:425
标识
DOI:10.1016/j.inffus.2023.101870
摘要

Image fusion aims to integrate complementary characteristics of source images into a single fused image that better serves human visual observation and machine vision perception. However, most existing image fusion algorithms primarily focus on improving the visual appeal of fused images. Although there are some semantic-driven methods that consider semantic requirements of downstream applications, none of them have demonstrated the potential of image-level fusion compared to feature-level fusion, which fulfills high-level vision tasks directly on multi-modal features rather than on a fused image. To overcome these limitations, this paper presents a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity constraints, termed PSFusion. First of all, the sparse semantic perception branch extracts sufficient semantic features, which are then progressively integrated into the fusion network using the semantic injection module to fulfill the semantic requirements of high-level vision tasks. The scene fidelity path within the scene restoration branch is devised to ensure that the fusion features contain complete information for reconstructing the source images. Additionally, the contrast mask and salient target mask are employed to construct the fusion loss to maintain impressive visual effects of fusion results. In particular, we provide quantitative and qualitative analyses to demonstrate the potential of image-level fusion compared to feature-level fusion for high-level vision tasks. With the rapid advancement of large-scale models, image-level fusion can expeditiously leverage the advantages of multi-modal data and state-of-the-art (SOTA) unimodal segmentation to achieve superior performance. Furthermore, extensive comparative experiments demonstrate the superiority of our PSFusion over SOTA image-level fusion alternatives in terms of visual appeal and high-level semantics. Even under harsh circumstances, our method offers satisfactory fusion results to serve subsequent high-level vision applications. The source code is available at https://github.com/Linfeng-Tang/PSFusion.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
LJH发布了新的文献求助10
1秒前
彭于晏的应助被露稚采纳,获得10
10秒前
13秒前
善良从露完成签到,获得积分10
16秒前
LJH发布了新的文献求助10
17秒前
Alfred发布了新的文献求助10
20秒前
LJH发布了新的文献求助10
24秒前
25秒前
Maestro_S的应助被科研通管家采纳,获得10
27秒前
molihuakai的应助被科研通管家采纳,获得10
27秒前
27秒前
2589发布了新的文献求助10
31秒前
炙热初丹完成签到,获得积分10
31秒前
LJH发布了新的文献求助10
33秒前
多情的井完成签到,获得积分10
39秒前
40秒前
42秒前
43秒前
47秒前
48秒前
49秒前
51秒前
54秒前
LJH发布了新的文献求助10
55秒前
55秒前
阳光元彤发布了新的文献求助10
56秒前
令狐大侠发布了新的文献求助30
57秒前
令狐大侠发布了新的文献求助10
57秒前
57秒前
专注凌柏发布了新的文献求助10
59秒前
令狐大侠发布了新的文献求助10
1分钟前
充电宝的应助被xccccccc采纳,获得10
1分钟前
令狐大侠发布了新的文献求助10
1分钟前
令狐大侠发布了新的文献求助10
1分钟前
令狐大侠发布了新的文献求助10
1分钟前
令狐大侠发布了新的文献求助10
1分钟前
LJH发布了新的文献求助10
1分钟前
Alfred完成签到,获得积分10
1分钟前
义气的夏旋完成签到,获得积分10
1分钟前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1700
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
八旗制度の研究 600
高分子基功能界面水蒸发材料与应用研究 500
外场增强的水汽发电研究 500
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8185137
求助须知:如何正确求助?哪些是违规求助? 9574052
关于积分的说明 21834040
捐赠科研通 7782575
什么是DOI,文献DOI怎么找? 3434520
关于科研通互助平台的介绍 2558743
邀请新用户注册赠送积分活动 2481206