Small Object Detection by DETR via Information Augmentation and Adaptive Feature Fusion

Small object detection algorithms face the challenge of achieving both accuracy and real-time performance. Although the RT-DETR model excels in real-time object detection, its performance degrades when detecting small objects. To address this, we propose two key improvements. First, RT-DETR currently relies solely on the last layer of backbone features for input to the transformer. However, this approach overlooks important detailed information such as edges and textures that are essential for detecting small objects. By introducing an information augmentation method, we enrich the input to the transformer with both semantic and detailed information, thus improving the accuracy of small object detection. Second, the RT-DETR decoder treats feature maps of different levels with equal weight, which doesn’t effectively handle the intricate relationships within multi-scale information. To overcome this, we introduce an adaptive feature fusion algorithm. This algorithm assigns learnable parameters to each feature map of different levels, enabling the model to adaptively fuse multi-scale feature maps and improve its ability to accurately capture object features across different scales. Experimental results show that our proposed model outperforms the latest DETR model, establishing a new state-of-the-art benchmark on the aquarium object detection dataset.