Monocular 3D vehicle detection with multi-instance depth and geometry reasoning for autonomous driving

Abstract We propose MCK-NET, a novel monocular framework for 3D target detection, location and pose estimation in autonomous driving scenarios, in which the key challenge is how to effectively mine depth information from a monocular image. To tackle this ill-posed problem, we combine the relative instance-depth of multiple corners in a monocular image to explicitly construct the corresponding depth relations between interest regions, from which MCK-NET learns to detect and locate objects based on geometric reasoning. In addition, there are two significant features existing in MCK-NET: One is to use the relative relationship between the 2D center and the 3D center projection to help locate the 3D center. The other is that the geometric constraints are established, including semantic keypoints, 2D box center and the projection of 3D center, which can enhance the 3D corner detection and improve the estimation accuracy of 3D centers. Experiments on the KITTI datasets show that MCK-NET achieves the most advanced results in all three tasks and outperforms the current monocular methods.