直线粒子加速器
变压器
计算机科学
计算机硬件
电气工程
工程类
电压
梁(结构)
土木工程
作者
Haonan Du,Chenyi Wen,Zhengrui Chen,Li Zhang,Qi Sun,Zheyu Yan,Cheng Zhuo
出处
期刊:Design, Automation, and Test in Europe
日期:2025-03-31
卷期号:: 1-7
被引量:7
标识
DOI:10.23919/date64628.2025.10992717
摘要
Nonlinear functions (NFs) in Transformers require high-precision computation consuming significant time and energy, despite the aggressive quantization schemes for other components. Piece-wise Linear (PWL) approximation-based methods offer more efficient processing schemes for NFs but fall short in dealing with functions with high nonlinearities. Moreover, PWL-based methods still suffer from inevitably high latency introduced by the Multiply-And-Add (MADD) unit. To address these issues, this paper proposes a novel quadratic approximation scheme and a highly integrated, multiplier-less hardware structure, as a unified method to accelerate any unary nonlinear function. We also demonstrate implementation examples for GELU, Softmax, and LayerNorm. The experimental results show that the proposed method achieves up to 5.41% higher inference accuracy and 60.12% lower area-delay product.
科研通智能强力驱动
Strongly Powered by AbleSci AI