A Generalization Sample Learning Method of Deep Learning for Semantic Segmentation of Remote Sensing Images

Deep learning methods have been widely studied in the semantic segmentation field of the remote sensing image. Training images play an important role in these methods; however, each training image usually contains not only the generalization information of each land category but also the specific interclass context between different categories. The specific interclass context prevents deep learning methods from focusing on generalization information learning during training and limits the performance on different data distributions. This article proposes a generalization sampling learning method of deep convolutional neural network (GSL-CNN) to emphasize generalization information learning for the semantic segmentation of remote sensing images. The proposed method develops a new CBR sampling strategy that contains three modules: category grouping ( $\mathbf {\boldsymbol {C}}$ ), basic unit extraction ( $\mathbf {\boldsymbol {B}}$ ), and random combination ( $\mathbf {\boldsymbol {R}}$ ). Module $\mathbf {\boldsymbol {C}}$ collects each land category map and strips away the specific interclass context from the raw annotated image. Module $\mathbf {\boldsymbol {B}}$ extracts basic units with different granularities from each land category map, and each basic unit can keep the generalization information of this category. Module $\mathbf {\boldsymbol {R}}$ aims to enhance the robustness against different data distributions by randomly picking basic units of different categories and randomly generating their interclass context. The new GSL-CNN method integrates the CBR sampling strategy with the convolutional neural network (CNN) model for semantic segmentation. Experiments on different remote sensing datasets and 15 state-of-the-art CNN models validated that the proposed method has the potential of improving the generalization ability of the CNN method from a sampling perspective.