CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding

Self-supervised learning (SSL) has gained widespread attention in the remote\nsensing (RS) and earth observation (EO) communities owing to its ability to\nlearn task-agnostic representations without human-annotated labels.\nNevertheless, most existing RS SSL methods are limited to learning either\nglobal semantic separable or local spatial perceptible representations. We\nargue that this learning strategy is suboptimal in the realm of RS, since the\nrequired representations for different RS downstream tasks are often varied and\ncomplex. In this study, we proposed a unified SSL framework that is better\nsuited for RS images representation learning. The proposed SSL framework,\nContrastive Mask Image Distillation (CMID), is capable of learning\nrepresentations with both global semantic separability and local spatial\nperceptibility by combining contrastive learning (CL) with masked image\nmodeling (MIM) in a self-distillation way. Furthermore, our CMID learning\nframework is architecture-agnostic, which is compatible with both convolutional\nneural networks (CNN) and vision transformers (ViT), allowing CMID to be easily\nadapted to a variety of deep learning (DL) applications for RS understanding.\nComprehensive experiments have been carried out on four downstream tasks (i.e.\nscene classification, semantic segmentation, object-detection, and change\ndetection) and the results show that models pre-trained using CMID achieve\nbetter performance than other state-of-the-art SSL methods on multiple\ndownstream tasks. The code and pre-trained models will be made available at\nhttps://github.com/NJU-LHRS/official-CMID to facilitate SSL research and speed\nup the development of RS images DL applications.\n