Application of a visual-based autonomous drone system for greenhouse muskmelon phenotyping

Abstract. Compared to traditional methods of monitoring crop growth or assessing health status using stationary sensors, drone systems offer the advantages of faster data collection and increased precision due to their flexible maneuverability. This study specifically focuses on developing a visual-based autonomous drone navigation system designed for localizing greenhouse melons. To enhance the stability of the drone navigation system and improve the accuracy of the melon localization algorithm, we made two key improvements. Firstly, we utilized ArUco markers as anchor points and integrated their detection into Enhanced ORB-SLAM2, a modified version of ORB-SLAM2 developed by us. This addition allows the system to detect and track ArUco markers, which serve as visual reference points within the greenhouse environment. Secondly, we calibrated the pre-built point cloud map to ensure its accuracy and alignment with the actual environment. By combining these enhancements, we achieved a more stable and precise drone navigation system for localizing greenhouse melons. The drone sends live RGB images to the ground control station, which runs ROS Melodic on the Ubuntu 18.04 operating system. The Enhanced ORB-SLAM2 algorithm uses the drone‘s images and a pre-built point cloud map to determine the drone's location within the greenhouse. Furthermore, in the greenhouse, the drone maintains a root mean square error of below 30 centimeters for three types of flight missions: straight line, closed-loop without turning, and closed-loop with turning. The melon tracking algorithm was built using the YOLOv4 object detection model and DeepSORT object tracking algorithm. To reduce ID switching, a three-step data cleaning method was applied to the tracking results, which proved to be significantly effective. Triangulation is utilized to calculate the positions of individual muskmelon fruits within the greenhouse. Through the implementation of two calibration methods, the melon position error has been effectively reduced to 0.223 meters. Finally, our system can analyze the quantity and positions of melons using the images recorded by the drone navigation. The experimental results confirm the viability of the system, and the proposed approach offers an efficient method for accurately locating melons within the greenhouse.