Multimodal Remote Sensing Object Detection Based on Prior-Enhanced Mixture-of-Experts Fusion Network

计算机科学 目标检测 遥感 传感器融合 融合 人工智能 对象(语法) 图像融合 计算机视觉 模式识别(心理学) 地质学 图像(数学) 语言学 哲学
作者
Kewei Liu,Dongliang Peng,Tao Li
出处
期刊:IEEE Transactions on Geoscience and Remote Sensing [Institute of Electrical and Electronics Engineers]
卷期号:63: 1-14 被引量:15
标识
DOI:10.1109/tgrs.2025.3585634
摘要

Multimodal remote sensing image object detection enhances detection accuracy by fusing complementary information from multimodal image data. However, complex environments significantly affect the reliability and complementarity of multimodal data, and traditional methods struggle to dynamically adapt to environmental changes, leading to degraded detection performance. To address this challenge, in this paper, we propose a multimodal remote sensing object detection method based on a prior information-enhanced mixture-of-experts fusion network. Specifically, we first introduce a prior information-enhanced mixture-of-experts fusion network framework to achieve environment-adaptive multimodal image feature fusion. Secondly, we propose a dynamic gating network that combines prior information and multimodal image features to endow the system with environmental perception capabilities. This network is employed to dynamically allocate weights to sub-fusion experts optimized for different environmental conditions within the mixture-of-experts fusion network framework. Furthermore, to fully exploit the complementary information present in multimodal image features, we propose a frequency-decoupled feature fusion network as a sub-fusion expert within the mixture-of-experts fusion network framework. This utilizes wavelet transform to decouple the features of each modality and then develops personalized fusion strategies for each frequency subband. In addition, to enhance detection efficiency, we introduce a cross-scale feature channel interleaved fusion strategy, which significantly reduces computational cost while ensuring stable detection performance. Experimental results on the DroneVehicle and RGBT-Tiny datasets demonstrate that our method achieves competitive performance compared to state-of-the-art approaches. Code will be available at: https://github.com/LiuKewei0110/MDPMFN.
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
红红发布了新的文献求助10
刚刚
Zengjx完成签到,获得积分10
刚刚
顶真完成签到,获得积分10
刚刚
csdhdhsiuhihi完成签到,获得积分10
刚刚
1秒前
长情诗蕾发布了新的文献求助10
1秒前
Paz完成签到,获得积分10
1秒前
补丁完成签到,获得积分10
1秒前
猫ovo猫完成签到,获得积分10
1秒前
1秒前
Cynthia完成签到,获得积分10
2秒前
老实皮皮虾完成签到,获得积分10
2秒前
2秒前
无花果的应助被kk采纳,获得10
2秒前
lalala发布了新的文献求助10
3秒前
yuyaoren完成签到,获得积分10
3秒前
l1t发布了新的文献求助30
3秒前
3秒前
hoke发布了新的文献求助10
4秒前
PostGGBone完成签到 ,获得积分10
4秒前
4秒前
ws完成签到,获得积分10
4秒前
4秒前
loy发布了新的文献求助20
4秒前
5秒前
weixuefeng完成签到,获得积分20
5秒前
zhangte完成签到,获得积分10
5秒前
lin完成签到,获得积分10
5秒前
Noontide完成签到,获得积分10
6秒前
共享精神的应助被看啥啥会采纳,获得10
6秒前
6秒前
冉冉完成签到,获得积分10
6秒前
6秒前
眼睛大的冰蓝完成签到,获得积分10
6秒前
7秒前
Jasper的应助被zhou采纳,获得10
7秒前
7秒前
敏感依丝完成签到,获得积分10
7秒前
7秒前
7秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1500
The World Atlas of Wine 9th edition 1000
奈特基础组织学:配套组织病理学(第4版)(Netter's Essential Histology With Correlated Histopathology) 800
John McDowells Bildungstheorie 800
八旗制度の研究 600
Analyses de reconnaissances de dettes passées devant les échevins d'Ypres (1249-1291) 600
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8159633
求助须知:如何正确求助?哪些是违规求助? 9555145
关于积分的说明 21725138
捐赠科研通 7749593
什么是DOI,文献DOI怎么找? 3427281
关于科研通互助平台的介绍 2552345
邀请新用户注册赠送积分活动 2471619