Geometric calibration for LiDAR-camera system fusing 3D-2D and 3D-3D point correspondences

Calibrating the extrinsic parameters on a system of 3D Light Detection And Ranging (LiDAR) and the monocular camera is a challenging task, because accurate 3D-2D or 3D-3D point correspondences are hard to establish from the sparse LiDAR point clouds in the calibration procedure. In this paper, we propose a geometric calibration method for estimating the extrinsic parameters of the LiDAR-camera system. In this method, a novel combination of planar boards with chessboard patterns and auxiliary calibration objects are proposed. The planar chessboard provides 3D-2D and 3D-3D point correspondences. Auxiliary calibration objects provide extra constraints for stable calibration results. After that, a novel geometric optimization framework is proposed to utilize these point correspondences, thus leading calibration results robust to LiDAR sensor noise. Besides, we contribute an automatic approach to extract point clouds of calibration objects. In the experiments, our method has a superior performance over state-of-the-art calibration methods. Furthermore, we verify our method by computing depth map and improvements can also be found. These results demonstrate that our method performance on the LiDAR-camera system is applicable for future advanced visual applications.