Point-pixel fusion for object detection and depth estimation

Object detection and recognition is a meticulous procedure that demands a great deal of accuracy when it comes to sophisticated autonomous motion, especially for the autonomous drive in an intricate environment. This paper introduces a point-pixel fusion system for object detection and classification with depth information for an autonomous vehicle system. Point-pixel fuses the lidar points from the 3D coordinates onto 2D pixels of the camera image, similarly camera image is also transformed into 2D pixels frames from the 3D environment. Further, YOLO4 is used to detect the object as a region of interest (ROI) and then lidar points are blended into ROI to determine the depth of the objects detected by the YOLO4 detector. Lidar point only in the ROI are kept for depth estimation, rest of the lidar points are repudiated. The experimental results demonstrated that the proposed early sensor fusion method is an efficient and reliable solution for object detection, classification, and depth estimation for an autonomous driving system.