ICG-VIMOD: infrared-characteristic-guided visible-infrared multi-modal object detection network

Abstract Object detection in complex lighting and harsh environments significantly benefits from the synergistic deployment of visible and infrared spectra. However, most existing dual-modality methods follow a two-stage pipeline termed ‘fusion-then-detection’, inevitably entailing substantial model complexity and intensive computational overhead. Moreover, existing enhancement strategies tend to bias toward visible imagery, lacking adequate adaptation to distinct infrared properties. To address these challenges, this paper proposes an infrared-characteristic-guided visible-infrared multi-modal object detection network (ICG-VIMOD). By integrating dual-modality feature fusion and detection within a unified architecture, an Infrared Position-aware feature Direct Connection mechanism is introduced to enable the end-to-end joint optimization, thereby maximizing the utility of infrared feature. Specifically, to mitigate the feature sparsity and imbalanced thermal distribution inherent in infrared images, two core components are designed: 1) An Edge-Guided Feature Extraction Module, which integrates the explicit boundary-capturing capability from edge operators with the adaptability of learnable convolutions to alleviate detail attenuation; And 2) A Region-adaptive Hybrid Attention mechanism, which dynamically allocates attention resources based on regional detection difficulty to improve infrared feature representation. Comprehensive benchmarks on the LLVIP, M3FD, and a self-built dataset validate the efficacy of our framework. On LLVIP and M3FD, ICG-VIMOD achieves a mAP@0.5 of 97.0% and 85.2%, and a mAP@0.5:0.95 of 67.0% and 57.9%, respectively. Featuring a mere 4.16 M parameters and 10.2G FLOPs, our model obtains improved detection accuracy while maintaining an ultra-lightweight footprint.