计算机科学
现场可编程门阵列
变压器
嵌入式系统
工程类
计算机硬件
电气工程
电子工程
数字信号处理
人工智能
微控制器
作者
Jiacheng Cao,Jiaqi Guo,Wei Xiong,Huanlin Luo,Jian Wang,Jinmei Lai
出处
期刊:
日期:2025-09-01
卷期号:: 227-235
被引量:1
标识
DOI:10.1109/fpl68686.2025.00040
摘要
Vision Transformer (ViT) has achieved great success in computer vision tasks, and FPGA-based ViT inference acceleration has recently gained widespread attention. However, the massive number of parameters and intensive matrix computations make it challenging to accelerate ViT models on resource-constrained FPGAs. To address this challenge, prior works have explored ViT quantization and approximate implementations of non-linear operations, but significant hardware resource consumption and potential performance optimization opportunities remain. In this paper, we propose an end-to-end quantized ViT accelerator, EQViTA. Its multi-kernel architecture and time-multiplexed scheduling strategy enable efficient resource utilization and memory-friendly features. We customize designs for the key operators of ViTs. First, we compress the model using INT4 quantization and implement the dataflow of the quantized model through resource-optimized dequantization. Second, we eliminate the convolution hardware module by using the convolution-to-linear mapping, thereby reducing resource usage. Finally, we achieve low-cost and highly parallel acceleration of self-attention and linear transformations through efficient exponential approximation for softmax and an adaptive linear transformation engine. Experiments on the Xilinx ZCU106 FPGA show that compared with state-of-the-art works, EQViTA achieves$1.03 \times$to$8.64 \times$improvements in energy efficiency and$1.14 \times$to$18.6 \times$improvements in normalized throughput. Meanwhile, EQViTA significantly reduces LUT, FF, BRAM, and DSP resource consumption, making it more suitable for resource-constrained FPGAs. Compared to the full-precision DeiT-Tiny model implemented with PyTorch, the INT4 quantized model deployed on EQViTA exhibits a 5.93 % accuracy drop.
科研通智能强力驱动
Strongly Powered by AbleSci AI