WaveFormer: Spectral–Spatial Wavelet Transformer for Hyperspectral Image Classification

Transformers have proven effective for Hyperspectral Image Classification (HSIC) but often incorporate average pooling that results in information loss. This paper presents WaveFormer, a novel transformer-based approach that leverages wavelet transforms for invertible downsampling. This preserves data integrity while enabling attention learning. Specifically, WaveFormer unifies downsampling with wavelet transforms to decompress feature maps without loss. This provides an efficient tradeoff between performance and computation. Furthermore, the wavelet decomposition enhances the interaction between structural and shape information in image patches and channel maps. To evaluate WaveFormer, we conducted extensive experiments on two benchmark hyperspectral datasets. Our results demonstrate that WaveFormer achieves state-of-the-art classification accuracy, obtaining overall accuracies of 95.66% and 96.54% on the Pavia University and the University of Houston datasets, respectively. By integrating wavelet transforms, WaveFormer presents a new transformer architecture for hyperspectral imagery that achieves superior classification without information loss from average pooling.