Obstacle Detection Method Based on Fusion of Visual and Millimeter-Wave Radar

The fusion scheme of millimeter-wave radar(MMW radar) and visual sensors combines the characteristics of strong penetration and immunity to lighting interference from MMW radar, as well as the high resolution and robust recognition capabilities of cameras, enabling accurate perception and identification of obstacles, thereby significantly enhancing the adaptability of autonomous driving systems in complex weather conditions and road scenarios. This paper presents an obstacle detection method based on the fusion of visual and MMW radar information. This method replace the traditional Region Proposal Network(RPN) in the Faster R-CNN framework with the approach that utilizes high-precision target distance and velocity information from MMW radar to generate multi-scale candidate bounding boxes. By training and testing the network model of the fusion algorithm using the nuScenes dataset, the feasibility and effectiveness of the proposed method are validated, showcasing its robust adaptability in complex environments.