S2DBFT: Spectral–Spatial Dual-Branch Fusion Transformer for Hyperspectral Image Classification

Convolutional neural networks (CNNs) and Transformer-based models have achieved remarkable success in hyperspectral image (HSI) classification tasks due to their outstanding ability to extract spatial and spectral features. However, most existing methods process spatial and spectral features separately, making it difficult to effectively learn their interactive features. To address this issue, we propose a spectral-spatial dual-branch fusion Transformer (S2DBFT) for HSI classification. Initially, we construct a spectral feature extraction module (SPEEM) and a spatial feature extraction module (SPAEM) to extract low-level features. These two modules consist of a one-dimensional convolution layer and a two-dimensional convolution layer, respectively, performing shallow extraction of spectral and spatial features. Next, the two feature sets obtained are fused through a weighted fusion process. Additionally, we design a multi-head spectral-spatial self-attention (MHS3A) mechanism to enhance the interactive fusion of spectral and spatial features. Upon completion of feature fusion, a linear layer is used to obtain the sample labels. Extensive experiments on four HSI datasets demonstrate the effectiveness of the proposed S2DBFT, compared to existing state-of-the-art methods. In terms of performance evaluation, the overall accuracy and average accuracy indicate the superiority and generalizability of S2DBFT.