Plant organ segmentation from point clouds using Point-Voxel CNN

Abstract. Plant phenotyping is a vital process in plant breeding studies and crop improvement. Automated segmentation of plant organs is a challenging task for conventional images due to occlusions. Three-dimensional (3D) point cloud data can overcome the challenge as the 3D data provides depth information and minimizes occlusion. The goal of this study is to perform plant organ segmentation from point clouds using 3D deep learning method called Point-Voxel Convolutional Neural Network (PVCNN). PVCNN utilizes point-based and voxel-based 3D data representation to achieve computational efficiency that results in higher speed up and lower memory consumption compared with other 3D deep learning approaches. We use cotton plants to predict main stem, branches, and cotton bolls. Point cloud data were collected using a LiDAR scanner. We manually annotated point clouds using Open3D library. After training, PVCNN achieved mIOU and accuracy of around 81% and 92.7%, respectively. After postprocessing, the segmented organs were used in extracting phenotypic traits. Comparison of ground truth and predicted segments demonstrated an R squared value of 0.9 for the main stem's height and diameter after excluding outliers. The number of nodes achieved an R squared value of 0.6 and a root mean square error of average internode distance was less than 0.1 cm. This plant organ segmentation method based on a 3D point-voxel CNN makes it possible to measure phenotypical traits related to plant architecture, which is important for plant breeding and physiology.