CTGAN : Cloud Transformer Generative Adversarial Network

Cloud occlusions obstruct some applications of remote sensing imagery, such as environment monitoring, land cover classification, and poverty prediction. In this paper, we propose the Cloud Transformer Generative Adversarial Network (CTGAN), taking three temporal cloudy images as input and generating a corresponding cloud-free image. Unlike previous work using generative networks, we design the feature extractor to maintain the weight of the cloudless region while reducing the weight of the cloudy region, and we pass the extracted features to a conformer module to find the most critical representations. Mean-while, to address the lack of datasets, we collected a new dataset named Sen2_MTC from the Sentinel-2 satellite and manually labeled each cloudy and cloud-free image. Finally, we conducted extensive experiments on FS-2, the STGAN dataset, and Sen2_MTC. Our proposed CTGAN demonstrates higher qualitative and quantitative performance than the previous work and achieves state-of-the-art performance on these three datasets. The code is available at https://github.com/come880412/CTGAN