UA-Fusion: Uncertainty-Aware Multimodal Data Fusion Framework for 3-D Object Detection of Autonomous Vehicles

In dynamic traffic scenarios, uncertainties such as occlusions, small objects, as well as unpredictable adverse weather conditions, prevent current environment perception methods from achieving the necessary levels of accuracy, reliability, and safety. In this article, an uncertainty-aware multimodal data fusion framework named UA-Fusion is proposed for 3-D multiobject detection (3D MOD). This framework aims to improve comprehensive and reliable perception capabilities in uncertainty-aware scenarios. Specifically, an uncertainty-aware fusion (UAF) decoder based on a probabilistic cross-modal attention mechanism (PCAM) is presented. This strategy explicitly models and leverages uncertainties in object prediction. It also allows for adaptive and complementary fusion of multimodal data, thereby addressing both aleatoric and epistemic uncertainty. Furthermore, an uncertainty-reduced object query initialization (UOQI) strategy is proposed. This approach fully utilizes the advantages of 2-D object detection in identifying small objects as priors to generate high-quality 3-D queries. Finally, a robustness optimization strategy for training based on query denoising (QD-ROST) is proposed to improve training robustness and convergence in the presence of uncertainty factors. Extensive experiments are conducted on the real-world dataset nuScenes. Notably, UA-Fusion effectively addresses challenges related to uncertainty. Additionally, experiments on the Argoverse 2 (AV2) and RADIATE datasets further validate the generalizability and effectiveness of the proposed method.