Visual Navigation and Crop Mapping of a Phenotyping Robot Mars-Phenobot in Simulation

AbstractCultivating high-yield and high-quality crops is important for addressing the growing demand for food and fiber from an increasing population. In selective breeding programs, autonomous robotic systems have proved to have great potential to replace manual phenotypic trait measurements which are time-consuming and labor-intensive. In this paper, we presented a Robot Operating System (ROS)-based phenotyping robot, MARS-PhenoBot, and demonstrated its visual navigation and field mapping capacities in the Gazebo simulation environment. MARS-PhenoBot was a solar-powered modular platform with a four-wheel steering and four-wheel driving configuration. We developed a navigation strategy that fuses multiple cameras to guide the robot to follow crop rows and transition between them, enabling visual navigation across the entire field without relying on global GNSS signals. Three row-detection algorithms, including thresholding-based, detection-based, and segmentation-based methods, were compared and evaluated in simulated crop fields with discontinuous and continuous crop rows, as well as with and without the presence of weeds. The results demonstrated that the segmentation-based method achieved the lowest average cross-track errors, measuring 2.5 cm for discontinuous scenarios and 0.8 cm for continuous scenarios in row detection. Additionally, a field mapping workflow based on RTAB-MAP (Real-Time Appearance-Based Mapping) and V-SLAM (Visual Simultaneous Localization and Mapping) was developed. The workflow produced the 2D maps identifying crop and weed locations, as well as 3D models represented as point clouds for crop shapes and structures. Using this mapping workflow, the average crop localization error was measured at 6.4 cm, primarily caused by the visual odometry drift. The generated point clouds of crops could support further phenotyping analyses, such as crop height/diameter measurements and leaf counting. The methodology developed in this study could be transferred to real-world robots that are capable of automated robotic phenotyping for in-field crops, providing an effective tool for accelerating selective breeding programs.