PSFormer: Pyramid Superpixel Transformer for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is a core processing procedure in the remote sensing community, which has been recently well studied using vision transformers (ViTs). However, due to the high computational and memory complexities, existing transformer-based classification methods tend to restrict the spatial extent of the transformer to small cropped HSI patches instead of the whole HSI data, thus sacrificing the essential strength of transformers in long-range interaction modeling and overlooking the beneficial multiscale features in HSI data. Inspiringly, here we propose PSFormer, a novel pyramid superpixel transformer (PSFormer) method specifically for HSI classification, in order to make full use of the transformer to excavate multiscale local-global features in HSI data. Specifically, a progressive superpixel merging strategy is introduced to flexibly control the scale of feature maps. Furthermore, a unique transformer backbone design based on a spectral attention layer and a classification head with a gate mechanism are developed, to adaptively exploit valuable local-global information at different scales with low computational cost. Extensive experimental results on five widely used datasets demonstrate the superiority of PSFormer over other state-of-the-art networks. For the sake of reproducibility, the related code of the PSFormer method will be open-sourced at: https://github.com/immortal13.