SWMCC: A Lightweight Network for Remote Sensing Change Caption

Remote sensing image change captioning (RSICC) aims to extract change information from bi-temporal remote sensing images using natural language, emerging as a new field of research in recent years. Model lightweighting is a crucial approach for facilitate real-world applications. Current studies primarily focus on enhancing model performance, while relatively limited attention has been paid to model lightweighting. To address this problem, we propose a novel lightweight and high-performance framework, SWMCC, which includes: 1) A dual-stream StarNet-based feature extractor for multi-scale feature extraction; 2) A Spatial Relationship Enhancement (SRE) module comprising a multi-scale difference mechanism to improve sensitivity to change regions and a high-dimensional spatial projection module leveraging differential features to boost decoder capabilities; 3) A Mamba-Trans architecture caption generator that efficiently decodes the correspondence between image embeddings and word embeddings. Experiments demonstrate the effectiveness of our proposed modules and the designed Mamba decoder. The SWMCC framework achieves the lowest inference costs, fastest inference speeds on the LEVIR-CC dataset, and excellent performance compared with state-of-the-art (SOTA) models. The code and pre-trained weights will be available at https://github.com/Fullahea/SWMCC.git.