PV-EncoNet: Fast Object Detection Based on Colored Point Cloud

Object detection is the most critical and foundational sensing module for the autonomous movement platform. However, most of the existing deep learning solutions are based on GPU servers, which limits their actual deployment. We present an efficient multi-sensor fusion based object detection model that can be deployed on the off-the-shelf edge computing device for the vehicle platform. To achieve real-time target detection, the model eliminates a large number of invalid point clouds through ground filtering algorithm, and then adds texture information (fused from camera image) through point cloud coloring to enhance features. The proposed PV-EncoNet efficiently encodes both the spatial and texture features of each colored point through point-wise and voxel-wise encoding, and then predicts the position, heading and class of the objects. The final model can achieve about 17.92 and 24.25 Frame per Second (FPS) on two different edge computing platforms, and the detection accuracy is comparable with the state-of-the-art models on the KITTI public dataset (i.e., 88.54% for cars, 71.94% for pedestrians and 73.04% for cyclists). The robustness and generalization ability of the PV-EncoNet for the 3D colored point cloud detection task is also verified by deploying it on the local vehicle platform and testing it on real road conditions.