FMAF-YOLO: fine-grained multi-modal adaptive fusion network for multispectral object detection

Abstract Multispectral object detection leverages the complementary characteristics of visible (RGB) and infrared (IR) modalities to improve target perception under low illumination, occlusion, and other complex environmental conditions. However, existing methods often suffer from imbalanced cross-modal fusion, leading to incomplete information integration and insufficient exploitation of fine-grained complementary cues. To address these limitations, we propose FMAF-YOLO, a lightweight dual-branch network for multispectral object detection. In the backbone, a C3k2-MLCA module preserves coarse spatial structure via local average pooling and introduces a global–local dual-branch attention mechanism to enhance the representation of small objects and edge regions. In the neck, a multi-scale cross-modal fusion module performs adaptive inter-layer interaction and weighted fusion of RGB and IR features to improve fusion balance and complementarity. In addition, a channel-aware dynamic sampling module jointly models spatial and channel information during upsampling, reducing feature blurring and preserving fine-grained details. Experiments on two public multispectral benchmarks demonstrate the effectiveness of the proposed method. On the M 3 FD dataset, FMAF-YOLO improves mAP@0.5 and mAP@0.5:0.95 by 4.1 and 3.7 percentage points, respectively, over the YOLO11n baseline. On the DroneVehicle dataset, evaluated under the official public train/validation/test protocol, the corresponding gains are 2.7 and 2.4 percentage points. These results demonstrate that FMAF-YOLO achieves effective fine-grained feature representation and adaptive cross-modal fusion while maintaining competitive computational efficiency.