Research on Multimodal Fusion Perception Technology for Autonomous Sweeping Vehicle

This paper proposes a multi-modal fusion framework to address the challenges of detecting and tracking specialized vehicles and dynamic targets in complex industrial park environments. The framework integrates LiDAR, a monocular camera, and an Inertial Navigation System (INS) to achieve precise obstacle perception and stable tracking through dynamic ROI (Region of Interest) cropping, optimized point cloud clustering, target detection, and multimodal perception fusion. First, a path-aware dynamic ROI cropping method and a multi-region density-aware seed point cloud ground segmentation approach are introduced to improve adaptability and point cloud processing efficiency. Second, a two-stage refinement strategy method is proposed to enhance target clustering accuracy. Furthermore, by combining the 2D detection network, a multimodal perception fusion module, and a multi-target tracking （MOT） strategy, the framework significantly improves fusion efficiency and matching accuracy. Field tests demonstrate that the framework achieves excellent performance, with static object localization deviations below 0.8 meters and reliable state estimation for dynamic targets. On a custom dataset, the monocular camera achieves 91.67% accuracy for specialized vehicles, while the fusion framework exhibits strong adaptability and reliability in complex scenarios.