An improved cascade R-CNN and RGB-D camera-based method for dynamic cotton top bud recognition and localization in the field

Cotton top cutting is an indispensable part of cotton planting. Cotton top bud detection and localization are highly challenging tasks because of tiny targets, dense growth and varying illumination. To achieve the automatic cutting of cotton buds in the field, a cotton top bud recognition and location algorithm adapted to a moving platform in the field based on images acquired by a red, green, blue and depth (RGB-D) camera was developed. In this study, an improved Cascade R-CNN network was proposed to detect cotton top bud regions on RGB images, and three-dimensional (3D) coordinates of targets were obtained by combining color images and depth images from RGB-D cameras. The 3D spatial position of the target in the world coordinate system was affected by the time of the cotton top bud recognition algorithm, the forward speed of the mobile platform and the time consumption of the manipulator moving to the target position. A dynamic compensation method of target coordinates in the moving direction was proposed to ensure the identification and positioning accuracy of cotton top buds in the moving process. To verify the effectiveness of the proposed algorithm, cotton top bud recognition and localization experiments were conducted in the field. The average precision with the proposed improved Cascade R-CNN model was 97.5 %, and the FPS was 13.3 frames per second, which was suitable for different period of a day. The average error of the positioning accuracy in the platform forwarding direction was 4.2 mm, 6.6 mm and 9.6 mm at speeds of 0.1 m/s, 0.2 m/s and 0.3 m/s, respectively. All the results demonstrate that the proposed method could be used for robotic cotton top cutting.