Top-tuning on transformers and data augmentation transferring for boosting the performance of weed identification

Detecting weeds at an early stage is crucial in reducing herbicide usage and preventing significant losses in agricultural productivity. The emergence of new computer vision techniques, such as transformers, presents promising opportunities for enhancing current in-field weed identification systems. Transformers, in comparison to Convolutional Neural Networks (CNNs), have demonstrated fewer biases toward textures and improved recognition of shapes, which are particularly relevant in weed identification where plant morphology is significant. In this study, two versions of Swin transformers were compared with EfficientNet-v2, a state-of-the-art CNN architecture. Weight transfer from ImageNet was employed, and data augmentation techniques from AutoAugment on the SVHN dataset were integrated into the proposed pipeline—this combination of transfer learning techniques aimed to mitigate the limitations of small agricultural datasets. The results of the large-sized Swin-v2 transformer, combined with transferred data augmentation, achieved a top-1 accuracy of 98.51% on the DeepWeeds dataset. Furthermore, a top-tuning stage was incorporated to enhance performance, reaching 98.61% accuracy. Precisely, the Softmax layer was removed, and Support Vector Machines and Gradient Boosting were trained on top of the bottleneck features. Finally, the Grad-CAM++ algorithm was utilized to compare the explanations of weed identifications before and after training. This analysis highlighted specific regions within the images that could be utilized for subsequent actions by robotic systems or other applications.