Learning Frequency-Domain Fusion for Multimodal Remote Sensing Semantic Segmentation

Multimodal remote sensing data substantially enhance semantic segmentation accuracy by providing complementary information across sensing modalities. However, fully exploiting and effectively fusing features from different modalities to capture comprehensive semantic representations remains a challenge. Most existing methods restrict interactions to the spatial domain, making their representations vulnerable to heterogeneity arising from distinct imaging mechanisms. To address these issues, we propose FDMF-Net, a Frequency-domain Decoupled Modality Fusion network. Our approach comprises three key modules: the Amplitude Spectrum Decoupling module (ASD), the Modality Enhancement module (ME), and the Low-Frequency-Guided Feature Fusion module (LFGF), dedicated to extracting, enhancing, and fusing modality-invariant and specific representations, respectively. The ASD module performs frequency-domain decomposition to separate modality-invariant and modality-specific features, promoting more effective cross-modal complementarity. The ME module introduces a mutual information-based feature enhancement scheme to obtain more robust modality-invariant and modality-specific representations, thereby improving feature discriminability. The LFGF module, based on an attention mechanism, fuses shared and specific representations to generate feature maps with richer semantic information. Extensive evaluations on multiple standard multimodal remote sensing datasets demonstrate that FDMF-Net achieves state-of-the-art accuracy across several benchmarks. The code is available at https://github.com/fy-sun/FDMF-Net.