Instance Segmentation on 3D City Meshes for Building Extraction

Digital twins are becoming increasingly popular in society for performing simulations. However, to conduct simulations, it is necessary to extract information about the objects composing an urban environment. Recently, a new semantic segmentation model applied to textured meshes, named PicassoNet-II, has been developed. The architecture of this model will be modified to perform segmentation of building instances rather than semantic segmentation. Additionally, a contextual analysis based on Markov fields is integrated into the algorithm to perform a contextual analysis of the features following segmentation. To train a 3D city segmentation model that can be generalized to any dataset, a large amount of annotated data is required. The model is trained using real data from Quebec City, Canada, as well as simulated data from different platforms such as Unreal Engine and Evermotion. Experimental results on semantic segmentation demonstrate that both simulated data and a Markov based analysis improves segmentation results overall.