DCTN: Dual-Branch Convolutional Transformer Network With Efficient Interactive Self-Attention for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is an essential task in remote sensing with substantial practical significance. However, most existing convolutional neural network (CNN)-based classification methods focus only on local spatial features while neglecting global spectral dependencies. Meanwhile, Transformer-based methods exhibit robust capabilities for global spectral feature modeling but struggle to extract local spatial features effectively. To fully exploit the local spatial feature extraction capabilities of CNN-based networks and the global spectral feature extraction capabilities of Transformer-based networks, this paper proposes a dual-branch convolutional Transformer method with efficient interactive self-attention for hyperspectral image classification, namely the dual-branch convolutional Transformer network (DCTN), which can aggregate local and global spatial-spectral features fully. Specifically, DCTN includes two core modules: the spatial-spectral fusion projection module and the efficient interactive self-attention module. The former utilizes 3D convolution with adaptive pooling and 2D group convolution with residual connection to parallel extract fused and grouped spatial-spectral features, respectively. The latter performs efficient interactive self-attention across height, width and spectral dimensions, enabling deep fusion of spatial-spectral features. Extensive experiments on three real HSI datasets demonstrate that the proposed DCTN method outperforms existing classification methods, yielding state-of-the-art classification performance. The code is available at https://github.com/AllFever/DeepHyperX-DCTN for reproducibility.