Asymmetric Dual-Encoder Network With Clustering and Mutual Contrast Loss for the Semantic Segmentation of Remote-Sensing Images

In recent years, the semantic segmentation of multimodal remote-sensing images using convolutional methods has received significant attention. Owing to the localized nature of convolutional operations, existing methods use the attention method to obtain global relations. However, effectively complementing and eliminating the global and local relations between two modalities has become a major challenge. In addition, category imbalance often affects the model performance as a result of the high resolution of remote-sensing images. In this study, we propose an asymmetric dual encoder network with clustering mutual contrast loss. Specifically, we use a convolutional neural network and Transformer as the backbone to extract two modal features in parallel to learn the local and global information, respectively. Next, our proposed multimodal hierarchical interaction module and dynamic weight inspired block efficiently fuse the multimodal features to complement the local and global information. The fused features are fed into our proposed local context extraction module and global context extraction module. Furthermore, to address the challenge presented by feature class imbalances, we apply a clustering algorithm to classify each pixel, which is subsequently inter-supervised using the inter-contrast loss. Extensive experiments on benchmark datasets show that the proposed model is extremely effective in the semantic segmentation of remotely sensed images and that it outperforms current state-of-the-art networks both quantitatively and qualitatively. Our code and results can be found at https://github.com/LYZ00918/AMCNet.