Convolutional neural network based on the fusion of image classification and segmentation module for weed detection in alfalfa

Abstract BACKGROUND Accurate and reliable weed detection in real time is essential for realizing autonomous precision herbicide application. The objective of this research was to propose a novel neural network architecture to improve the detection accuracy for broadleaf weeds growing in alfalfa. RESULTS A novel neural network, ResNet‐101‐segmentation, was developed by fusing an image classification and segmentation module with the backbone selected from ResNet‐101. Compared with existing neural networks (AlexNet, GoogLeNet, VGG16, and ResNet‐101), ResNet‐101‐segmentation improved the detection of Carolina geranium, catchweed bedstraw, mugwort and speedwell from 78.27% to 98.17%, from 79.49% to 98.28%, from 67.03% to 96.23%, and from 75.95% to 98.06%, respectively. The novel network exhibited high values of confusion matrices (>90%) when trained with sufficient data sets. CONCLUSION ResNet‐101‐segmentation demonstrated excellent performance compared with existing models (AlexNet, GoogLeNet, VGG16, and ResNet‐101) for detecting broadleaf weeds growing in alfalfa. This approach offers a promising solution to increase the accuracy of weed detection, especially in cases where weeds and crops have similar plant morphology. © 2024 Society of Chemical Industry.