CNN-TransNet: A Hybrid CNN-Transformer Network With Differential Feature Enhancement for Cloud Detection

Thin clouds detection and the difficulty in distinguishing between clouds and bright surface features have consistently presented challenges in optical remote sensing cloud detection tasks. Convolutional neural networks (CNNs) have made significant progress, however, CNNs perform weakly in capturing global information interactions due to the inherent limitation of network structure. To address these issues, we propose a hybrid CNN-Transformer network with differential feature enhancement (DFE) for cloud detection (CNN-TransNet). CNN-TransNet adopts a dual-branch encoder consisting of CNN-Transformer module and DFE module. CNN-TransNet combines the strengths of both Transformer and CNN to enhance finer details and build long-range dependencies. CNN is considered as a high-resolution feature extractor for capturing low-level features. The transformer module encodes image sequences by patch embedding to extract high-level features and relationships. DFE branch utilizes differential features and attention mechanism to further obtain effective information for distinguishing between clouds and non-clouds. The decoder upsamples features of the encoder and concatenates multiscale features from the CNN layers. Experimental results demonstrate that the proposed method achieves excellent performance on Landsat-8 and Sentinel-2 images, with a high cloud pixel precision of 92.94% and 93.04%. Moreover, it effectively reduces thin cloud omissions and the misclassifications of bright surface features.