OPT-SAR-MS2Net: A Multi-Source Multi-Scale Siamese Network for Land Object Classification Using Remote Sensing Images

The utilization of optical and synthetic aperture radar (SAR) multi-source data to obtain better land classification results has received increasing research attention. However, there is a large property and distributional difference between optical and SAR data, resulting in an enormous challenge to fuse the inherent correlation information to better characterize land features. Additionally, scale differences in various features in remote sensing images also influence the classification results. To this end, an optical and SAR Siamese semantic segmentation network, OPT-SAR-MS2Net, is proposed. This network can intelligently learn effective multi-source features and realize end-to-end interpretation of multi-source data. Firstly, the Siamese network is used to extract features from optical and SAR images in different channels. In order to fuse the complementary information, the multi-source feature fusion module fuses the cross-modal heterogeneous remote sensing information from both high and low levels. To adapt to the multi-scale features of the land object, the multi-scale feature-sensing module generates multiple information perception fields. This enhances the network’s capability to learn contextual information. The experimental results obtained using WHU-OPT-SAR demonstrate that our method outperforms the state of the art, with an mIoU of 45.2% and an OA of 84.3%. These values are 2.3% and 2.6% better than those achieved by the most recent method, MCANet, respectively.