Extrinsic Calibration of a Binocular Camera and LiDAR Based on Neural Networks

LiDAR–binocular camera systems have attracted increasing research attention with their advantages of two sensors. The two-sensor data fusion premise accurately calibrates the external parameters of both sensors. However, existing LiDAR and binocular camera calibration uses geometric methods to calibrate the left and right cameras with the LiDAR or register the LiDAR point cloud and camera depth map to obtain the extrinsic parameters. Existing methods have difficulty obtaining low binocular camera baseline length and reprojection errors simultaneously. This article proposes a binocular camera and LiDAR extrinsic parameter calibration method based on a back propagation (BP) neural network and checkerboard. First, a LiDAR–binocular camera calibration model is established, and the extrinsic parameters and intermediate variables to be calibrated are determined. The calibration rotation matrix has multiple variables that are not independent of each other, creating a complex model. The nine rotation matrix variables are decoupled and reduced using the Euler angle representation. Then, the BP neural network structure is determined using the external parameter calibration data characteristics. The calibration algorithm accuracy is improved by introducing the baseline and reprojection errors into the loss function. Finally, we use real data to check the calibration performance and introduce the baseline length and reprojection errors as indicators to improve the calibration result evaluation accuracy and effectiveness. The results show that the proposed calibration method has good consistency and small baseline and reprojection errors. Compared with traditional perspective-n-point (PnP) methods, the average baseline error is reduced by 64.804%, and the reprojection error is reduced by approximately 10%.