PI-Net: Point-to-Image Knowledge Distillation for Camera-based 3D Semantic Scene Completion

Camera-based Semantic Scene Completion (SSC) aims to infer the geometric structure and semantic information in the entire 3D scene from limited 2D images. However, due to the lack of geometric information in the image, existing methods tend to generate fuzzy completion and incorrect semantic boundaries. In this paper, we propose cross-modal knowledge distillation to address this issue, namely PI-Net, which guides the camera-based model to learn accurate 3D geometry to compensate for spatial surroundings information during training. Specifically, we propose a point cloud occupancy prediction model as the teacher, leveraging its output for strong depth supervision signals and spatial voxel information to enhance the student model. To facilitate effective distillation, we design depth guidance distillation to improve geometric predictions, and spatial guidance distillation to assist the student model in better capturing the structural information of the surrounding environment. Finally, prediction domain distillation is incorporated to facilitate holistic learning from point cloud to image. Experimental results demonstrate that PI-Net outperforms state-of-the-art camera-based methods on challenging benchmarks—SemanticKITTI and SSCBench-KITTI-360.