A Pseudo-LiDAR and Image Fusion Method for Autonomous Driving

Autonomous vehicles typically integrate LiDAR sensors for precise but sparse distance measurements and camera sensors for rich visual data, albeit without 3D object location. This paper introduces a fusion algorithm designed to incorporate pseudo-LiDAR information generated from RGB images with LiDAR data and image edges. Firstly, the approach extracts depth estimates from RGB images to produce dense yet somewhat noisy pseudo-LiDAR data. Subsequently, it establishes connections between pseudo-LiDAR points and LiDAR points, incorporating image edges for denser and more accurate fusion data. The resultant fused data can then be utilized as input for other 3D object detection networks, yielding enhanced performance in multi-sensor fusion perception. Experimental results on the KITTI benchmark corroborate the effectiveness of the proposed method.