C2DFF-Net for Object Detection in Multimodal Remote Sensing Images

Multimodal fusion detection has proven to be a practical approach for improving small target perception in remote sensing. Existing studies primarily focus on the fusion of multimodal spatial-domain features, while paying insufficient attention to cross-domain differential features. Furthermore, exploration of lightweight fusion mechanisms tailored for deployment on edge devices such as UAVs remains relatively limited.To address these issues, we propose an extremely lightweight cross-modal and cross-domain differential feature fusion network (C²DFF-Net). The network includes three innovative plug-and-play modules. Specifically, we propose a cross-modal differential feature interaction (CDFIM) module to facilitate inter-modal information interaction by enhancing differential features between visible and infrared modalities. To improve the adaptive fusion of multimodal complementary features, we propose a cross-domain gated self-attention (CGSA) module. The global context information of the image is explored from a frequency-domain perspective, and a polarized self-attention mechanism is introduced to establish long-range dependencies on spatial-frequency domain differential features while filtering redundant information. Additionally, we design an adaptive light-aware mask (ALM) module to enable the network to learn effective multimodal complementary features without bias, improving its robustness in complex lighting environments such as exposure and glare. Comprehensive experiments demonstrate that C²DFF-Net not only achieves state-of-the-art (SOTA) performance but also maintains a lightweight design, achieving 85.7% mAP50 with 6.58M parameters and 14.6 GFLOPs on the DroneVehicle dataset. Furthermore, it shows a favorable accuracy-computation tradeoff compared to SOTA models on the VEDAI and FLIR datasets. Finally, we conduct real-world deployment experiments, and C²DFF-Net achieves satisfactory results under various lighting conditions, clearly demonstrating its practical applicability. Code will be available at https://github.com/FPGAzzy/C2DFF-Net.