AVT: A New Asymmetric Vision Transformer Architecture

Transformer excel at modeling long-range dependencies in sequential data, and their integration into computer vision has challenged the long-standing dominance of CNN(Convolutional Neural Network) in image processing. In this paper, we address the poor performance of vision Transformer on small-sample image tasks by proposing an asymmetric Vision Transformer architecture: query and$\text{key}(\mathrm{Q}$and K) are produced by shallow networks, while value (V) are generated by a deeper network. This design rebalances the allocation of modeling resources between global and local representations, shifting computational emphasis during training toward local feature extraction. Experimental results demonstrate that the proposed asymmetric Vision Transformer substantially improves Transformer performance on small-sample datasets.