Omni-Scan2BIM: A ready-to-use Scan2BIM approach based on vision foundation models for MEP scenes

Mechanical, electrical, and plumbing (MEP) systems play a crucial role in providing various services and creating comfortable environments for urban residents. In order to enhance the management efficiency of these highly complex MEP systems, as-built building information models (BIMs) are being increasingly adopted worldwide. As-built BIMs accurately represent the actual conditions of facilities, making as-built BIM reconstruction significantly important for construction progress tracking, quality assurance, subsequent facility management, and renewal. To create as-built BIMs for MEP systems, laser scanners are widely utilized to capture high-resolution images and dense 3D measurements of the environment in a fast and highly accurate manner. Despite research efforts to automatically achieve "Scan-to-BIM," there are still gaps in applying current solutions to real-world scenarios. One of the major challenges are the limited generalization of existing methods to unseen scenarios without proper training or fine-tuning on custom-designed datasets. To address this issue, this study introduces Omni-Scan2BIM, a novel approach powered by large-scale pre-trained vision foundation models. Omni-Scan2BIM enables the recognition of MEP-related components with a single shot by integrating an all-purpose feature extraction model and a class-agnostic segmentation model. Firstly, given only a single image with a reference mask, the visual features are extracted for both the target component and the collected on-site images using the vision foundation model DINOv2. Secondly, through comparing pixel features, similarity maps are generated for the on-site images. The prior points for the class-agnostic segmentation model are sampled from the local maxima of the similarity map. Thirdly, Segment Anything Model (SAM) is leveraged to sequentially segment the target component. Finally, the target component is segmented out in 3D space based on the transformation matrix describing the spatial relationship between the 2D images and the 3D point clouds. Shape analysis and label fusion are conducted for as-built BIM modeling purpose. To validate the feasibility of the proposed technique, experiments were conducted using data collected from a real construction site in Hong Kong. The results demonstrate that the proposed Omni-Scan2BIM approach can easily generalize to unseen components with significantly improved accuracy and efficiency.