Automating graph-based geometric digital model generation for building digital twin applications from point cloud and image data

Creating geometric digital twins of buildings remains a labor-intensive process, often limited to the reconstruction of structural elements. Non-structural components and their spatial relationships with indoor spaces are rarely integrated into a unified digital representation. This paper proposes a novel semi-automated method for generating graph-based geometric digital models for digital twins from 3D point cloud and image data. The approach extracts spatial and object information from these data based on deep learning methods. Multiple 2D detectors are trained on different public and customised datasets to broaden class coverage and their predictions are mapped into a unified label space, fused per view, projected into 3D space, and merged into object instances that correspond to the same physical element. A new graph schema is then introduced to represent indoor spaces and elements, capturing both hierarchical and spatial relationships. The schema links each entity to its geometric representation and supports temporal snapshots. All extracted information is structured and stored in a graph database using the proposed schema. The method is validated on two real-world datasets, one residential house and one institutional facility, capturing a broader and more differentiated range of object classes across different building types and topologies. The promising results indicate that the method has the potential to be generalized to a wider range of buildings. • Propose a novel method to construct graph-based representation of indoor building environments utilizing point cloud data and image data. • Identify room spaces and detect non-structural elements such as furniture and electrical components via multi-detector, multi-view fusion. • Design a graph schema that captures hierarchical spatial relationships, links to geometric point cloud data and provides temporal snapshots for tracking object changes over time. • Represent extracted elements and their relationships using graph structures for efficient data storage and analysis. • Demonstrate the method on real-world point cloud data for validation.