RCM-depth: depth estimation based on camera and sparse radar expansion

Abstract In this paper, we propose an innovative radar point cloud dynamic enhancement method based on attribute features, combined with a Transformer model to effectively learn the correlation between radar point clouds and images. Existing depth estimation methods underutilize radar attributes like local density and depth deviation. To address this, we propose RCM-Depth, a two-stage network that dynamically adjusts radar point clouds for accurate spatial representation. This network dynamically adjusts the radar point clouds to more accurately reflect their true spatial relationships. First, we dynamically expand the radar point clouds using intrinsic feature information, increasing their density and accuracy to enhance spatial representation. Second, through the Transformer model, we capture the deep relationship between radar point clouds and images, achieving high-quality multimodal data fusion. Additionally, we introduce a module called RSA, which reuses features discarded during the fusion phase and incorporates more accurate confidence prior information, thereby significantly improving the accuracy of depth estimation. We validated this method on the nuScenes benchmark, achieving a 17.42% reduction in mean absolute error and an 18.67% reduction in root mean square error compared to existing state-of-the-art methods. These results demonstrate the effectiveness of our method in improving the quality and accuracy of depth estimation, showcasing its advantages and practicality in handling complex multimodal scenarios.