Instance Segmentation and Berry Counting of Table Grape before Thinning Based on AS-SwinT

Berry thinning is one of the most important tasks in the management of high-quality table grapes. Farmers often thin the berries per cluster to a standard number by counting. With an aging population, it is hard to find adequate skilled farmers to work during thinning season. It is urgent to design an intelligent berry-thinning machine to avoid exhaustive repetitive labor. A machine vision system that can determine the number of berries removed and locate the berries removed is a challenge for the thinning machine. A method for instance segmentation of berries and berry counting in a single bunch is proposed based on AS-SwinT. In AS-SwinT, Swin Transformer is performed as the backbone to extract the rich characteristics of grape berries. An adaptive feature fusion is introduced to the neck network to sufficiently preserve the underlying features and enhance the detection of small berries. The size of berries in the dataset is statistically analyzed to optimize the anchor scale, and Soft-NMS is used to filter the candidate frames to reduce the missed detection of densely shaded berries. Finally, the proposed method could achieve 65.7 APbox, 95.0 AP0.5box, 57 APsbox, 62.8 APmask, 94.3 AP0.5mask, 48 APsmask, which is markedly superior to Mask R-CNN, Mask Scoring R-CNN, and Cascade Mask R-CNN. Linear regressions between predicted numbers and actual numbers are also developed to verify the precision of the proposed model. RMSE and R2 values are 7.13 and 0.95, respectively, which are substantially higher than other models, showing the advantage of the AS-SwinT model in berry counting estimation.