HeightFusion: A Lightweight and Balanced Multi-Modal BEV Perception Framework for Real-Time 3D Object Detection

Multi-sensor fusion is essential for achieving accurate and reliable 3D perception in autonomous driving systems. While LiDAR-camera fusion methods significantly improve detection accuracy by combining geometric and semantic cues, most existing approaches rely on dense point cloud processing and complex 3D backbones, resulting in substantial computational overhead and high inference latency. These limitations hinder their deployment on resource-constrained platforms such as lightweight unmanned vehicles and edge devices. In this work, we observe that raw LiDAR point clouds contain redundant spatial information, much of which can be effectively com-pressed-particularly along the vertical dimension. Inspired by this insight, we propose HeightFusion, a lightweight and efficient multi-modal 3D object detection framework. HeightFusion encodes sparse LiDAR data into a compact 2D height map to preserve essential vertical structure while drastically reducing processing cost. The encoded LiDAR features are then fused with camera-based BEV features through a streamlined cross-modal fusion pipeline. Experiments conducted on the nuScenes benchmark demonstrate that HeightFusion achieves a high detection accuracy of 69.6% NDS while maintaining a low inference latency of 45 ms per frame. Overall, HeightFusion provides a highly efficient and accurate solution for real-time 3D perception in autonomous systems with constrained computational resources.