3D Point Cloud Segmentation Leveraging Global 2D-View Features

AbstractWith the introduction of widely available depth cameras and the increasing demand for digitalizing our physical world, there is now more than ever a need for an automated segmentation system to process the 3D point cloud data. The latest developments in this field utilize graph neural networks (GNN), but do not take globally consistent features into account, which could aid in carrying out robust segmentation predictions across entire scenes.In this paper, we propose a 3D point cloud segmentation method leveraging globally consistent features for improved performance. These global features are retrieved from a 2D fully convolutional network (U-Net) and then propagated to a 3D DGCNN together with raw point clouds for segmentation. Our segmentation based on globally coherent feature embedding space is novel and promising to perform classification and separation within entire scenes rather than in single blocks. Both quantitative and qualitative evaluations on widely known ScanNet and S3DIS datasets with a multitude of indoor RGBD scenes have proven the feasibility of considering global 2D-view features in 3D segmentation tasks with improved performance.KeywordsSemantic segmentationInstance segmentationPoint cloud analysisGraph neural networksGlobal features