Upgrading swin-B transformer-based model for accurately identifying ripe strawberries by coupling task-aligned one-stage object detection mechanism

With the wave of agricultural modernization, deep learning technology has brought revolutionary changes to the vision system of strawberry picking robots. Yet, the morphological diversity of strawberries, small and dense targets, and high overlap scenes make the detection and ripeness classification of strawberries a great challenge. To solve these problems, we introduce a new task-aligned one-stage object detection (TOOD) mechanism. Firstly, we incorporate the Swin-B (Swin-Base) transformer module to enhance the feature extraction performance in the backbone network. Secondly, we replace the original feature pyramid network (FPN) with CARAFE-FPN, which utilizes advanced upsampling methods to enhance detection at different scales. A multi-scale training (MST) approach is applied to capture the small targets effectively. Additionally, the Augmentations library is utilized for dataset augmentation to enhance the model's generalization. Lastly, we refine the task alignment learning head and propose simple anchor alignment metric (S-aam) to reduce the impact of parameters on network performance for finding the optimal solutions. We collected a complex strawberry image dataset of more than 90,000 instances to test the method effectiveness in detecting strawberry ripeness. The results show that our model achieves 74.1% average precision (AP), 93.9% AP50, and 84.1% AP75, respectively. Our model shows the superior detection performance compared to most of models with fewer parameters and lower FLOPs. In addition, our model obtained the highest accuracy in detecting small strawberry targets. To prove the generalization of the model, we also verify it on COCO dataset, and the results show that the performance has been enhanced by 0.7% compared to the baseline. In summary, our proposed methods can be used to accurately identify ripe strawberries, which has the potential to be applied in strawberry picking robot system.