SF-Mamba: A Semantic-Flow Foreground-Aware Mamba for Semantic Segmentation of Remote Sensing Images

Mamba-based methods are used in the semantic segmentation of remote sensing images (RSI) due to their advanced capacity in long-range modeling. However, its capacity for foreground object awareness is underutilized. A Semantic-flow Foreground-aware Mamba (SF-Mamba) is proposed to improve this capacity and semantic segmentation performance. The encoder of SF-Mamba is a dual-branch architecture; one branch extracts features through local modeling, and the other provides global information through a visual state space model (VSSM). A Foreground-aware decoder (FAD) is designed to learn object information. A semantic flow aggregation module (SFAM) is presented to cope with the complex scenes of remote sensing images. The feature matcher in SFAM enhances segmentation performance and reduces background noise while maintaining efficiency. Global-guided foreground awareness module (GFAM) is designed to improve foreground modeling capacity by calculating the similarity between each layer feature map and the global feature map. Experiments conducted on three datasets demonstrate that the SF-Mamba is more effective and efficient than other mainstream approaches. Visualization results and ablation experiments demonstrate the superiority in improving the semantic segmentation ability and dealing with complex remote sensing images, especially for small objects in RSI. The code is available at https://github.com/TiezhuXing01/SF-Mamba.