Modality-inter fusion and enhancement network for dual-spectral object detection

In object detection tasks, when facing situations such as low contrast, occlusion, multiple overlapped targets, and changes in lighting, the detection performance of pure visible light (RGB) images or infrared (IR) images for object detection is not good. Combining image information from both visible light and infrared modalities can result in higher accuracy and more robust object detection under these challenges. The key to building an object detector based on the visible light and infrared modalities is how to merge the two modalities to obtain a more effective feature representation. For this purpose, we first constructed feature fusion module and feature enhancement module to effectively fusion and enhance the feature representations of the two modalities of visible light and infrared. Second, by combining the Bottleneck Attention Module (BAM) and the feature fusion and enhancement module, a dual-spectrum object detection network is constructed. Finally, we conducted experiments on the LLVIP and FLIR datasets, achieving mean Average Precision (mAP) of 46.2% on FLIR and 68.1% on LLVIP, surpassing the results of other recent methods. Experimental results showed that the dual-spectrum object detection network constructed in this paper effectively improved the performance of object detection.