Vision Spiking Transformer for Image Classification

Vision Transformer (ViT) has achieved remarkable success in image classification by modeling longrange dependencies with self-attention. However, its quadratic computational complexity and dense feed-forward layers lead to high energy consumption, limiting deployment on low-power application. Spiking Neural Network (SNN) offers event-driven, sparse computation inspired by Leaky Integrate and Fire Neuron (LIF), which is advantageous for energy-efficient neuromorphic hardware but often suffers from reduced accuracy due to training issues. To achieve both high performance and efficiency, this work proposes hybrid architectures that integrate spiking mechanism into ViT. Three variants are implemented and evaluated: (1) introducing spiking neurons into feed-forward modules, (2) replacing the attention mechanism with spiking attention, and (3) extending spiking mechanism to patch embedding. Experiments on CIFAR-10 under identical training settings demonstrate that the neuron-augmented model achieves the best trade-off, improving accuracy to 89.63% with reduced loss of 0.8358. In contrast, spiking attention decreases accuracy to 86.35% (loss 0.9583), and spiking patch embedding further lowers performance to 84.38% (loss 1.0255). These results represent that partial integration of spiking mechanisms can enhance performance, but excessive spiking replacement reduces representational capacity. This study highlights the potential of selective spiking integration in ViT and provides insights into future hybrid neuromorphic architectures.