Mixed Residual Convolutions with Vision Transformer in Hyperspectral Image Classification

In recent years, deep learning methods represented by convolutional neural networks (CNN) have gradually become a research focus in the field of hyperspectral image classification (HSI). Although the proposed CNN-based methods have the advantages of spatial feature extraction, they are difficult to handle the spectral feature. We propose a mixed residual convolutions with Vision Transformer model (MRViT), which uses ViT Transformer (ViT) to overcome this limitation. First, using principal component analysis (PCA) and a channel shift strategy to construct a module to process HSI data. Then, a mixed residual convolution is constructed to extract spectral-spatial features. Finally, the important feature information is enhanced by the ViT model. In this paper, four datasets are used for experimental analysis, which confirms that the performance of MRViT model is superior to other HSI classification models. The research content of this paper enriches related research in HSI classification, and also provides certain reference value for subsequent research.