Cross-modal fusion for joint classification of hyperspectral and LiDAR images

In the field of remote sensing, efficient fusion of hyperspectral images (HSI) and light detection and ranging (LiDAR) data can capture comprehensive surface features encompassing spectral and elevation information, thereby enhancing classification performance. However, existing methods face challenges such as significant inter-modal feature discrepancies, insufficient utilization of global contextual information and lack of multi-dimensional interaction in attention mechanisms, etc. To address these issues, this study proposes a hybrid attention based fusion network (HAFNet). Specifically, the self-interactive attention module (SIAM) is designed to model long-range dependencies among spectral, spatial, elevation and cross-modal features, adaptively enhancing global feature representation and overcoming the limitations of traditional methods in global context modelling. The spatial expansion attention module (SEAM) is introduced to optimize the weight distribution of multimodal features and achieve fine-grained control of feature fusion by focusing on key regions. The cross-modal interaction (CMI) block is presented to establish deep feature correlations between HSI and LiDAR modalities such that complementary information across modalities can be efficiently utilized. Experiments on three publicly available HSI-LiDAR benchmark datasets including MUUFL Gulfport, Trento and Houston demonstrate the effectiveness and superiority of the proposed method.