A Hierarchical Vision-Language Model-Guided Feature Fusion Framework for Referring Remote Sensing Image Segmentation

Referring remote sensing image segmentation (RSRIS) aims to achieve target-oriented, fine-grained understanding of geospatial information by leveraging both visual and linguistic modalities. Different from traditional remote sensing semantic segmentation, RSRIS needs to address more complex contextual relationships and pronounced scale variations inherent to remote sensing imagery, which pose significant challenges for precise alignment and fusion between textual semantics and visual features. To tackle these issues, we propose a novel framework, termed VSPNet, for vision-language guided remote sensing referring segmentation. Specifically, VSPNet adopts a hybrid backbone architecture based on CNN, Transformer, and Mamba models to extract rich, multi-dimensional visual representations, thereby enabling more effective cross-modal interaction. Furthermore, we design a hierarchical multimodal feature fusion strategy tailored for RSRIS: at the shallow feature level, a Text-guided Texture Interaction Module (TTIM) is introduced to enhance the integration of fine-grained texture details with textual cues; at the deep feature level, a Text-guided Semantic Fusion Module (TSFM) is developed to facilitate global contextual alignment between segmentation targets and semantic expressions in language. Extensive experiments on public benchmarks, RefSegRS and RRSIS-D, demonstrate that VSPNet consistently outperforms state-of-the-art methods. Comprehensive ablation studies further verify the necessity and effectiveness of each constructed component. The code is available at https://github.com/NWPUFranklee/VSPNet.git.