An automatic 3D tomato plant stemwork phenotyping pipeline at internode level based on tree quantitative structural modelling algorithm

Phenotypic traits of stemwork are important indicators of plant growing status, contributing to multiple research domains including yield estimation, breeding engineering, and disease control. Traditional plant phenotyping with human work faces serious bottlenecks on labour intensity and time consumption. In recent years, the application of Quantitative Structural Modeling (QSM) together with three-dimensional (3D) sensor-based data acquisition techniques provides a feasible solution towards the automatic stemwork phenotyping. Nevertheless, existing QSM-based pipelines are sensitive towards the point cloud quality, and mostly focus on the phenotyping at plant or organ level. Information at internode level which are closely related to photosynthesis and light absorption was generally overlooked. To this end, a 3D automatic stemwork phenotyping pipeline is developed for tomato plants at both plant and internode level. Coloured point clouds are taken as the sensor input of the pipeline. A semantic segmentation based on PointNet++ was used to detect and localise the stemwork points. To improve the quality of the segmented stemwork point clouds, a density-based refining pipeline is proposed containing three main processes: non-replacement resampling, interference branch removal, and noise removal. A Tree Quantitative Structural Modeling (TreeQSM) algorithm was then applied to the stemwork point cloud to construct a digital reconstruction. The target phenotypic traits were finally calculated from the digital model by employing an internode association process. The proposed phenotyping pipeline was evaluated with a test dataset containing three tomato plant cultivars: Merlice, Brioso, and Gardener Delight. The related rooted mean squared errors of calculated internode length, internode diameters, leaf branching angle, leaf phyllotactic angle, and stem length range from 4.8 to 64.4%. Considering the time consuming manual phenotyping process, the proposed work provides a feasible solution towards the high throughput plant phenotyping, from which facilitates the related research on plant breeding and crop management.