Multiscale adaptive fusion network for joint classification of hyperspectral and LiDAR data

With the increasing availability of remote sensing data sources, the joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has become an active research area in earth observation tasks. However, current fusion methods still face challenges in cross-modal feature learning and multi-modal feature fusion. Specifically, insufficient cross-modal learning and simplistic feature concatenation limit multi-modal representation learning. To solve these problems, we propose a multiscale adaptive fusion network (MSAFNet). First, spectral-spatial features are extracted from HSI using a spectral-spatial residual cascaded feature extraction module, which simultaneously enhances the spectral feature representation. Second, multiscale information is extracted and fused using a hierarchical multiscale feature fusion module, which employs hierarchical fusion strategies to effectively enhance the representation of multiscale features. Third, elevation features are extracted from LiDAR data by using an elevation residual cascaded feature extraction module. Finally, considering the heterogeneity and complementarity of multi-modal features, a complementary multi-modal attention fusion module is developed to effectively fuse spectral-spatial and elevation features from different modalities via mutual injection. Moreover, it enhances the model’s classification performance through dynamic weight allocation. Extensive experiments on three widely used HSI and LiDAR datasets confirm that our proposed MSAFNet exhibits remarkable classification performance compared to several representative state-of-the-art methods.