AiRs: Adapter in Remote Sensing for Parameter-Efficient Transfer Learning

Remote sensing is stepping into the era of the foundation model, where the fine-tuning paradigm is widely adopted to transfer the profound knowledge of pretrained foundation models to downstream tasks. However, the full fine-tuning method would become inefficient in terms of training and storage, as the foundation models are getting larger and larger. Recently, a lot of deep learning research has proposed various parameter-efficient fine-tuning (PEFT) methods that perform well with a few trainable parameters. However, most of them focus on fine-tuning general foundation models without considering the special properties of remote sensing. In this article, we propose an adapter in remote sensing (AiRs) to fine-tune large foundation models for remote sensing downstream tasks by introducing the adapter-tuning framework. Specifically, we construct AiRs from two aspects: more expressive adaptation modules and a more efficient integration strategy. Specialized adaptation modules are applied to different functional layers in AiRs, which encode the inductive bias of remote sensing images and enhance the semantic concepts of geography. Moreover, AiRs establishes pathways between trainable modules with residual connections, which reduces training difficulty and improves performance. We conduct extensive experiments on object detection, semantic segmentation, and scene classification tasks. By training only 4.4% parameters of the pretrained backbone, AiRs surpasses the previous state-of-the-art (SOTA) PEFT competitors on all experimental datasets and outperforms the full fine-tuning on six out of ten datasets.