DReAM-Fusion: an RGB-T object detection network driven by the dual-modal recalibrated feature aggregation module

Abstract RGB-T object detection, by fusing complementary information from visible and infrared images, has wide applications in fields such as military reconnaissance, autonomous driving, and intelligent surveillance. Cross-modal feature fusion is a key component of this task. However, current approaches struggle with effective spatial-channel feature extraction and adaptive cross-modal calibration. By failing to fully exploit inter-modal complementarities, these methods often suppress key semantics or retain low-level noise, ultimately degrading the quality of the fused representation. To address these issues, this paper proposes a fusion network, DReAM-Fusion, driven by a dual-modal recalibrated feature aggregation module (DReAM). In DReAM, a channel emphasis convolution is designed to suppress redundant channels and reinforce key semantic information, while a channel-spatial collaborative attention mechanism is introduced to weight features across two dimensions, thereby enhancing the response of core information. Additionally, a shared group normalization detection head is designed, utilizing a shared convolution structure to reduce the feature distribution discrepancy between different detection layers. A learnable channel scaling layer is also incorporated, assigning trainable scaling factors to each channel, thus alleviating gradient instability issues. Lastly, we design a reparameterized multi-scale convolution (RMC). By incorporating dilated reparameterized convolution, RMC enhances scale adaptability while maintaining a constant computational cost through structural reparameterization. Experimental results show that DReAM-Fusion achieves mAP 50 of 87.7% and mAP of 60.8% on the M3FD dataset. Compared to the current state-of-the-art model, cross-modality fusion transformer (CFT), the proposed method reduces the parameter count by 27.59 M (62%) while still improving detection performance by 0.6% and 5.4% points, respectively. Furthermore, experiments on the KAIST and VEDAI datasets further validate the comprehensive advantages of the proposed method in terms of detection accuracy and computational efficiency.