计算机科学
变压器
计算机硬件
工程类
嵌入式系统
Softmax函数
电子工程
故障排除
电气工程
现场可编程门阵列
可重组计算
作者
Bang He,Bangxin Qin,Han Wang,Chenyang Dai,Lin Yang,Li Li,Yuxiang Fu
标识
DOI:10.1109/tvlsi.2026.3706708
摘要
The efficient implementation of the softmax is critical for optimizing transformer hardware accelerators. Unlike its role as a static, one-time classifier in convolutional neural networks (CNNs), softmax in transformers is core to achieving dynamic contextual awareness, generating attention weights that enable the model to focus on relevant information. Its high-frequency, large-scale execution across every layer makes the softmax a critical performance bottleneck, particularly when processing long sequences. Existing hardware implementations face two primary challenges: excessive resource consumption from the exponential (EXP) computation, and high latency induced by the softmax’s nonelementwise nature. This article proposes a domain-unified piecewise EXP approximation technique. This technique reduces the resource consumption of the EXP unit by 23% and its approximation error by 26%. Furthermore, the proposed design provides direct and efficient hardware-level support for the online softmax algorithm, significantly reducing latency. Under the TSMC 40-nm process, it achieves a$1.36\times $improvement in area-normalized throughput. On a Xilinx KCU116 FPGA, LUT-normalized throughput improves by$1.26\times $. Moreover, the proposed reconfigurable architecture substantially improves resource utilization. Finally, end-to-end evaluations confirm that the design strictly preserves native inference accuracy while delivering a notable 4.86% overall system-level latency reduction for Llama3-8B.
科研通智能强力驱动
Strongly Powered by AbleSci AI