Hybrid SNN-Transformer Network With Membrane-Potential-Aware Attention for Hyperspectral Image Classification

Vision Transformers (ViTs) achieve high accuracy in hyperspectral image classification (HSIC) yet incur prohibitive computational cost, hindering edge deployment. Spiking neural networks (SNNs) promise energy-efficient event-driven computation, but directly transferring ViTs to SNNs leads to severe information loss from binary spikes, which is further amplified under the extremely limited labeled samples typical of HSIC. To address these intertwined challenges, we propose a spiking grouped separable convolutional vision transformer (Spiking-GSCViT) featuring a novel membrane-potential-aware grouped self-attention (MPA-GSA) module. MPA-GSA calculates attention weights from continuous, differentiable membrane potentials, rather than from lossy binary spikes. Moreover, leveraging the time-varying nature of membrane potentials, this novel attention mechanism is able to attend to different spectral–spatial regions across multiple time steps, thereby preserving rich discriminative features. Extensive experiments on four benchmark datasets with only 1% to 5% of the labeled samples per class show that Spiking-GSCViT matches or surpasses its ANN counterpart (94.16% on Indian pines and 97.71% on Salinas), with only 0.21 M parameters. The consecutive MAC operations are reduced by 81%, and the spike-driven synaptic operations (SOPs) are decreased by 39.2%. As a lightweight SNN-ViT hybrid architecture, Spiking-GSCViT simultaneously mitigates the data-hungry nature of both ViTs and SNNs, enabling accurate and energy-efficient HSIC under limited supervision.