Cloud Removal With SAR-Optical Data Fusion Using a Unified Spatial–Spectral Residual Network

Cloud contamination greatly limits the potential utilization of optical images for geoscience applications. An effective alternative is to extract data from synthetic aperture radar (SAR) images to remove clouds due to the strong penetration ability of microwaves. In this paper, we propose a novel unified spatial-spectral residual network that utilizes SAR images as auxiliary data to remove cloud from optical images. The method can better establish the relationship between SAR and optical images and be divided into two modules: feature extraction and fusion module, and reconstruction module. In the feature extraction and fusion module, a gated convolutional layer is introduced to discriminate cloud pixels from clean pixels, which makes up for the lack of distinguishing ability of vanilla convolutional layers and avoids the error of cloud areas in feature extraction. In the reconstruction module, spatial and channel attention mechanisms are introduced to obtain global spatial and spectral information. The network is tested on three datasets with different spatial resolutions and compositions of land covers to verify the effectiveness and applicability of the method. The results show that the method outperforms other mainstream algorithms that simultaneously use SAR images as auxiliary data with a gain about 2.3 dB in terms of PSNR on SEN12MS-CR dataset.