3D Detection and Tracking for On-road Vehicles with a Monovision Camera and Dual Low-cost 4D mmWave Radars

High resolution 4D millimeter wave radar has been increasingly used for robust 3D detection and tracking of on-road vehicles. Rich point clouds generated by 4D radars can not only provide more reliable detection in harsh weather environments, but also offers 3D tracking capabilities for on-road objects. In this paper, a convolutional neural network (CNN) with cross fusion strategy is proposed for 3D on-road vehicle detection. The trained CNN model was also tested with dual low-cost 4D millimeter wave radars and a single monovision camera. An extended version of radar-camera calibration in three dimensions and 3D tracking with an extended Kalman filter (EKF) were also presented. The detection results showed that the proposed convolutional neural network model outperformed the one used on the Astyx dataset which provided up to 1500 radar detection points, on average, per frame.