RNAF-YOLO: a model for highlighting difficult-to-detect targets in remote sensing images

Abstract Challenges like scale variations, shape diversity, complex backgrounds, and sample imbalance in remote sensing images make some targets difficult to detect. Consequently, a model called RNAF-You Only Look Once (YOLO) is proposed, combining a composite convolution module with an Intersection over Union (IoU)-based weighted loss function for remote sensing object detection. A composite convolution module, RepFocalNet, is designed and incorporated into the backbone network to replace the original C2f layer, enhancing multi-scale modeling and feature extraction capabilities in complex backgrounds. Adaptive spatial correlation pyramid attention is introduced after the ninth layer, enhancing sensitivity to subtle features and improving small object detection. Furthermore, Focal Inner Soft IoU is designed to replace the original loss function. Ablation experiments were conducted on the DIOR dataset to verify the effectiveness of each module. Following this, the proposed method was compared with several leading methods to further evaluate its performance. Compared to the YOLOv8 model, RNAF-YOLO improved mAP@50 by 1.5% and increased recall by 3%. Additionally, the classification accuracy for the bridge and vehicle categories increased by 4.2% and 8.9%. Compared to other methods, RNAF-YOLO demonstrates superior performance across multiple classification accuracy metrics. As a consequence, the proposed method demonstrates superior performance in remote sensing object detection, effectively highlighting difficult-to-detect targets.