ConvFormerSR: Fusing Transformers and Convolutional Neural Networks for Cross-Sensor Remote Sensing Imagery Super-Resolution

Super-resolution (SR) techniques based on deep learning have a pivotal role in improving the spatial resolution of images. However, remote sensing images exhibit ground objects characterized by diverse types, intricate structures, substantial size discrepancies, and noise. Simultaneously, variations in imaging mechanisms, imaging time, and atmospheric conditions among different sensors result in disparities in image quality and surface radiation. These factors collectively pose challenges for existing SR models to fulfill the demands of the domain. To address these challenges, we propose a novel cross-sensor SR framework (ConvFormerSR) that integrates transformers and convolutional neural networks (CNNs), catering to the heterogeneous and complex ground features in remote sensing images. Our model leverages an enhanced transformer structure to capture long-range dependencies and high-order spatial interactions, while CNNs facilitate local detail extraction and enhance model robustness. Furthermore, as a bridge between the two branches, a feature fusion module (FFM) is devised to efficiently fuse global and local information at various levels. Additionally, we introduce a spectral loss based on the remote sensing ratio index to mitigate domain shift induced by cross-sensors. The proposed method is validated on two datasets and compared against existing state-of-the-art SR models. The results show that our proposed method can effectively improve the spatial resolution of Landsat-8 images, and the model performance is significantly better than other methods. Furthermore, the SR results exhibit satisfactory spectral consistency with high-resolution (HR) images.