HCMA-Net: Hierarchical Cross-Modality Aggregation Network for Multimodal Remote Sensing Image Classification

While multimodal remote sensing images provide complementary information in different imaging ways, effectively aggregating and jointly learning from these heterogeneous features is non-trivial, primarily due to the inherent modality gap that hinders semantic alignment and feature interoperability. To overcome these issues, we propose a Hierarchical Cross-Modality Aggregation Network (HCMA-Net). It introduces two novel components: the hierarchical feature aggregation and enhancement module (HFAE-Module) and the cross-modality interactive feature extraction module (CMIFE-Module). The HFAE-Module tackles the modality gap and enables cross-scale interaction through its Hierarchical Cross-Modality Feature Aggregation (HCMFA) mechanism, which incorporates Cross-Spectral and Spatial Aggregation Non-Local Attention layers (CSNLA and SANLA) to align features and aggregate contextual information across spectral and spatial dimensions. The CMIFE-Module addresses the optimization conflict by leveraging a dual-attention design; it uses self-attention to reinforce intra-modal coherence and cross-attention to dynamically extract and fuse complementary inter-modal features, thereby maximizing complementarity while avoiding the dilution of discriminative features and preventing negative transfer. Experiments on four real-world datasets (Hohhot, Nanjing, Xi’an, Houston2013) demonstrate that HCMA-Net consistently achieves outstanding classification results. The code is available at: https://github.com/sun740936222/HCMA-Net.