Multiscale Adapter Based on SAM for Remote Sensing Semantic Segmentation

The segment anything model (SAM), despite its remarkable performance in dense visual tasks, encounters a significant challenge in remote sensing image segmentation due to the intricate, multiscale objects and vast landscapes present in remote sensing imagery. To address this challenge, this article introduces a parameter-efficient fine-tuning approach that integrates multiscale adapters into the SAM image encoder for remote sensing image segmentation. By harnessing SAM's global modeling capabilities and marrying it with multiscale feature hierarchies, our proposed method maintains a consistent channel capacity and resolution throughout the entire network, thereby mitigating textural information loss resulting from spatial resolution downgrades. Furthermore, these adapters facilitate the interaction of features from regions of varying sizes, enabling the perception of features at multiple scales. Extensive experiments conducted on five benchmark remote sensing segmentation datasets demonstrate that our proposed method achieves state-of-the-art performance while significantly reducing the number of optimized parameters, highlighting its effectiveness and efficiency.