A Stepwise Refining Image-Level Weakly Supervised Semantic Segmentation Method for Detecting Exposed Surface for Buildings (ESB) From Very High-Resolution Remote Sensing Images

Exposed surface for buildings (ESB), which refers to exposed surfaces with traces of building construction, often leads to urban dust. Accurate ESB detection is important for planning urban development and improving urban environment. Fine-grained monitoring of ESB typically needs massive high-quality pixel-level labels, which are demanding and expensive. In contrast, obtaining cost-efficient image-level labels is more promising. Most image-level weakly supervised methods can extract pixel-level pseudo labels using the class activation map (CAM) generated by the classification network. Subsequently, these labels are applied to train the semantic segmentation network. However, the CAM is easy to miss fine-grained information, which leads to label noise. Moreover, the downsampling in the segmentation networks will further loss the spatial information. Furthermore, the sparse distribution and irregular shape of ESB pose additional challenges. Given these problems, we propose a stepwise refining image-level weakly supervised semantic segmentation method (SRIWS): 1) we introduce a new data augmentation method called SRMix to oversample the classification dataset; 2) we propose a two-branch network with a superpixel pooling layer (SPNet) as the semantic segmentation network to capture both global semantic information and spatial details; and 3) to alleviate the impact of potential noise in the initial labels, we design the high-confidence sample filtering operation (HSF) during the SPNet training. The evaluation experiments for the SRIWS were performed on three datasets. The results confirm that our proposed SRIWS presents a superior performance in recognizing ESB compared with existing state-of-the-art methods. In addition, numerous ablation experimental results indicate the effectiveness and robustness of our SRIWS.