CWRepViT-Net: An encoder-decoder deep learning framework with RepViT blocks for crop weed semantic segmentation in soybean fields through their life journey

With the world's population growing at a steady rate, agricultural product demand is significantly increasing. To meet this expanding need, it is critical to employ cutting-edge knowledge and technologies. Remote sensing data in conjunction with computer vision algorithms, plays an emerging and increasingly important role in precision weed management, which is essential for food security and safety. Drone images are one of the most accessible and efficient remote sensing data that can be collected in field crop production. In this study, drone images were captured at six intervals during June and July 2024 (i.e., 21, 26, 33, 39, 45, and 52 days after seeding (DAS))during soybean vegetative growth phases. By employing a Deep Learning (DL) network, we performed the crop vs. weed semantic segmentation on these captured images. The data set employed in this research was from a soybean experiment field that included five classes: soil, soybean crops, volunteer canola, other broadleaf weeds, and volunteer wheat plus other grassy weeds. The proposed semantic segmentation method is based on an encoder-decoder architecture, where the encoder path uses RepViT blocks, and the decoder path uses Modified UNet (MUNet) blocks. Since the network is trained to segment soybean crops, volunteer canola and other weed species, the proposed method is referred to Crop-Weed-RepViT-Net (CWRepViT-Net). This study outlines a five-step framework designed to segment crops and weeds in soybean fields using drone imagery. The steps consist of: (1) pre-processing image data, (2) training the segmentation network, (3) performing semantic segmentation of crop and weed classes, (4) evaluating model performance, and (5) applying the trained network to early-season soybean growth stages. . The CWRepViT-Net model achieved an overall accuracy of over 95% and a Kappa coefficient of 0.91, indicating strong agreement between predicted and actual labels and confirming the method’s effectiveness for early-stage crop and weed differentiation.