Superpixel-based time-series reconstruction for optical images incorporating SAR data using autoencoder networks

Time-series reconstruction for cloud/shadow-covered optical satellite images has great significance for enhancing the data availability and temporal change analysis. In this study, we proposed a superpixel-based prediction transformation-fusion (SPTF) time-series reconstruction method for cloud/shadow-covered optical images. Central to this approach is the incorporation between intrinsic tendency from multi-temporal optical images and sequential transformation information from synthetic aperture radar (SAR) data, through autoencoder networks (AE). First, a modified superpixel algorithm was applied on multi-temporal optical images with their manually delineated cloud/shadow masks to generate superpixels. Second, multi-temporal optical images and SAR data were overlaid onto superpixels to produce superpixel-wise time-series curves with missing values. Third, these superpixel-wise time series were clustered by an AE-LSTM (long short-term memory) unsupervised method into multiple clusters (searching similar superpixels). Four, for each superpixel-wise cluster, a prediction-transformation-based reconstruction model was established to restore missing values in optical time series. Finally, reconstructed data were merged with cloud-free regions to produce cloud-free time-series images. The proposed method was verified on two datasets of multi-temporal cloud/shadow-covered Landsat OLI images and Sentinel-1A SAR data. The reconstruction results, showing an improvement of greater than 20% in normalized mean square error compared to three state-of-the-art methods (including a spatially and temporally weighted regression method, a spectral–temporal patch-based method, and a patch-based contextualized AE method), demonstrated the effectiveness of the proposed method in time-series reconstruction for multi-temporal optical images.