Fast and Effective: Progressive Hierarchical Fusion Classification for Remote Sensing Images

Multisource remote sensing image fusion classification aims to produce accurate pixel-level classification maps by combining complementary information from different sources of remote sensing data. Existing methods based on Convolutional Neural Networks (CNN-based) utilize a patch-based learning framework, which has a high computational cost, leading to poor real-time performance. In contrast, methods based on Fully Convolutional Networks (FCN-based) can process the entire image directly, achieving fast inference. However, FCN-based methods require high computational resources and exhibit shortcomings in feature fusion, hindering practical applications. In this paper, a lightweight FCN-based Progressive Hierarchical Fusion Network (PHFNet) is tailored for multisource remote sensing image classification. PHFNet comprises a pyramid dual-path encoder and a pyramid decoder. In the encoder, cross-source features are hierarchically fused via the adaptive modulation fusion module (AMF), which leverages style calibration for cross-source alignment and promotes the complementarity of the fusion feature. In the decoder, we introduced an improved convolutional gated recurrent unit (iConvGRU) to progressively integrate the semantic and detailed information of hierarchical features, producing a context-enhanced global representation. In addition, we consider the relation between the channel number, convolutional kernel size, and parameter count to make the model as lightweight as possible. Comprehensive evaluations on three multisource remote sensing datasets demonstrate that PHFNet improves overall accuracy by 1.5% to 2.8% with a low computational overhead compared to state-of-the-art methods. The source code is avaliable at https://github.com/ShirlySmile/PHFNet .