Twice matched fruit counting system: An automatic fruit counting pipeline in modern apple orchard using mutual and secondary matches

Fruit counting, as one of the essential parts of yield estimation, is an important factor in production process planning. In the case of apple crops, it is useful in orchard management and as guidance for farmers, showing a decisive role in product market strategies and cultivation practices. Although some machine vision based studies have exhibited notable fruit counting ability, they still need to be improved for clustered fruit. This study proposes an automatic fruit counting pipeline called twice matched fruit counting system to overcome this limitation. The twice matched fruit counting system consists of three sub-algorithms: i) object detection model based on You Only Look Once Version 4-tiny; ii) fruit tracking with mutual match; iii) and fruit counting with ID assignment. The object detection model was developed based on You Only Look Once Version 4-tiny, which quickly and accurately detect fruit and trunks with mean average precision of 96.4% and detection speed of 16 ms. The fruit tracking with mutual match was designed to alleviate match errors associated with the clustered fruit, which achieved superior performance with average ID Switch Rate of 3.9%, Multiple Object Tracking Accuracy of 89.9% and Multiple Object Tracking Precision of 93.5%. The fruit counting was implemented by ID assignment, where each fruit was assigned with a unique ID based on fruit tracking results and direction of camera motion. The root mean squared error and coefficient of determination were 16.3 fruit per video and 0.93, respectively, which indicate a high correlation between fruit count results from the proposed approach and ground truth counting results. The twice matched fruit counting system was implemented on Central Processing Unit at 3–5 frames per second. These results demonstrate a potential of the twice matched fruit counting system for estimating fruit yield in modern apple orchards, which could provide technical support for orchard management.