Deep learning-based extraction of building contours for large-scale 3D urban reconstruction

Automated 3D reconstruction of urban models from satellite images remains a challenging research topic, with many interesting outcoming applications, such as telecommunications and urban simulation. To reconstruct 3D city from stereo pairs of satellite images, semi-automatic strategies are typically applied, which are based either on procedural modeling, or on the use of both image processing and machine learning methods to infer scene geometries together with semantics. In both cases, human interaction still plays a key role, in particular for the rooftop buildings extraction. In the last decade, the use of deep learning algorithms, notably convolutional neural networks (CNNs), has shown a remarkable success for automatic image interpretation. We propose an approach using CNN architecture to automatize the procedure of building contour extraction with the final purpose to automatize 3D urban reconstruction chain and improve the quality of the generated city models. The developed algorithm consists of three steps: 1) We apply a mask-based normalization technique to the input image. 2) CNN network is applied to obtain a raster map of buildings. 3) A polygonization algorithm is designed, which processes a raster map of building to output an ensemble of building contours. We have adopted a U-Net neural network for building segmentation task. We compare the use of several U-Net architectures with the purpose to retain the best suited model. To train models, we have built a dataset of high-resolution satellite images over 15 different cities, and the corresponding building masks. The experimental results show that the proposed approach succeeds in predicting building polygons in a short time, and exhibits good generalization properties to be applied on diverse Earth areas. The developed algorithm combined with the existing LuxCarta reconstruction chain improves 3D urban scene modeling results, and thus supplies an important step towards the automatic reconstruction of 3D city scenes.