Bridging the Gap: A Heterogeneous Collaborative Learning Framework via Frequency-Aware Synergy for Multimodal Remote Sensing Semantic Segmentation

Multimodal remote sensing image segmentation remains challenging due to cross-modal information gaps, background noise interference, and the inherent limitations of single-architecture feature learning. To address these issues, we propose a heterogeneous collaborative learning (HetCoLearn) framework that facilitates robust feature interaction via frequency-aware synergy. HetCoLearn comprises two complementary networks: the frequency-domain Gaussian multimodal network (FGMNet), which integrates a Fourier fusion module to capture cross-modal frequency correlations and employs an attention-based Gaussian filter for noise suppression, and the multiscale wavelet-based Manhattan network (MWMNet), which utilizes wavelet-based feature extraction to capture hierarchical spatial details and a Manhattan spatial-correlation module for precise edge detection. A hierarchical collaborative strategy enhances knowledge transfer through three components: 1) frequency-spatial feature alignment, which synchronizes global frequency patterns with local wavelet components; 2) collaborative weight correction within attention optimization; and 3) a consistency-guided decoding constraint that enforces semantic agreement between heterogeneous branches. Extensive experiments on the Vaihingen and Potsdam datasets validate the effectiveness of the HetCoLearn framework. Our approach outperforms several state-of-the-art methods, demonstrating its efficacy and robustness in complex remote sensing scenarios. The code and results can be found athttps://github.com/QshHub/HetCoLearn