Tomato cluster detection and counting using improved YOLOv5 based on RGB-D fusion

Accurate estimation of tomato cluster yields is critical to the advancement of intelligent and unmanned greenhouses, guiding horticultural management and adjusting sales and marketing strategies. However, due to the complex natural environment and tracking stability, there are still considerable challenges for automated yield estimation to be deployed in practice. Therefore, this paper presents an improved tomato cluster counting method that combines object detection, multiple object tracking, and specific tracking region counting. To reduce background tomato misidentification, we proposed the YOLOv5-4D that fuses RGB images and depth images as input. Next, we adopted ByteTrack to track tomato clusters in continuous frames and designed a specific tracking region counting method to overcome the problem of tracked tomato cluster ID shift. In the test set, the improved YOLOv5-4D had a detection accuracy of 97.9 % and a [email protected]:0.95 of 0.748. Field experiments showed that the counting method achieved a statistical average counting accuracy of 95.1 % and the integrated algorithm ran at more than 40 FPS, enabling stable real-time yield estimation.