FMA Is (Almost) All You Need: Sharing FMA Resources for Linear and Nonlinear Function Acceleration in Attention-Based Networks

加速度 非线性系统 计算机科学 功能(生物学) 数学 算法 控制理论(社会学) 数学优化 理论(学习稳定性) 应用数学
作者
Arpan Suravi Prasad,Andrea Belano,Davide Rossi,Francesco Conti,Luca Benini
出处
期刊:Proceedings [Institute of Electrical and Electronics Engineers]
卷期号:: 33-40
标识
DOI:10.1109/arith69512.2026.00014
摘要

Attention-based networks, namely Transformers and variants, are widely adopted across modern AI workloads due to their strong generalization capabilities. While execution in attention-based networks is dominated by linear operations that are efficiently accelerated by specialized matrix multiplication units, the growing diversity of nonlinear operations such as activation functions, softmax, and layer normalization has emerged as a key performance and energy bottleneck. Existing solutions mostly rely on dedicated nonlinear hardware units or software emulation, which incur high area or performance overhead. In this work, we present a fused-multiply-add (FMA)-centric approach for efficient nonlinear execution. Our approach leverages the abundant FMA units already present in GEMM accelerators to approximate a wide range of nonlinear functions, including reciprocal and inverse square root, using reconfigurable piecewise polynomial approximation (PWPA), without compromising native GEMM performance. We demonstrate the effectiveness of this approach by extending an open-source GEMM accelerator, enabling the efficient execution of softmax, a wide range of activation functions (GELU, SiLU, etc.), and layer normalization with low area overhead (10%). Implemented in TSMC 7 nm technology node and operating at 1 GHz, we achieve a peak throughput of 3.9, 3.9, and 7.8 GOp/s for softmax, layer normalization, and activation functions; corresponding energy efficiencies of 92, 98, and 172 GOp/J. Our flexible, low-overhead approach improves area efficiency by$\mathbf{1. 7} \times$and energy efficiency by$\mathbf{1. 4 6} \times$for softmax compared to existing work.

科研通智能强力驱动
Strongly Powered by AbleSci AI
科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
香蕉觅云的应助被11采纳,获得10
1秒前
1秒前
1秒前
1秒前
2秒前
向连虎发布了新的文献求助10
2秒前
2秒前
2秒前
亮司完成签到,获得积分20
2秒前
ddd发布了新的文献求助10
2秒前
3秒前
4秒前
4秒前
4秒前
Ava的应助被现阶段回电话采纳,获得10
4秒前
Mnipwefjgj311发布了新的文献求助10
4秒前
WeL完成签到,获得积分10
5秒前
liukang172发布了新的文献求助50
6秒前
川川发布了新的文献求助10
6秒前
超帅的诗槐的应助被王志远采纳,获得30
7秒前
Lucas的应助被WangJiarou采纳,获得10
7秒前
7秒前
我姓阿叫杜关注了科研通微信公众号
7秒前
小甜甜完成签到,获得积分10
8秒前
珍珠完成签到,获得积分10
8秒前
初景发布了新的文献求助50
8秒前
李健的应助被nanjiren采纳,获得10
8秒前
9秒前
9秒前
HopeStar发布了新的文献求助20
9秒前
淼淼完成签到 ,获得积分10
10秒前
10秒前
11秒前
张张张发布了新的文献求助10
12秒前
崎路人完成签到,获得积分10
12秒前
Vanilla657发布了新的文献求助30
12秒前
12秒前
13秒前
跳跃奇异果完成签到 ,获得积分10
13秒前
13秒前
高分求助中
(应助此贴封号)【重要!!请各用户(尤其是新用户)详细阅读】【科研通的精品贴汇总】 10000
拉伸高分子熔体中缠结行为的模拟和小角中子散射研究 1500
John McDowells Bildungstheorie 800
Analyses de reconnaissances de dettes passées devant les échevins d'Ypres (1249-1291) 600
Handbook of Pharmaceutical Excipients. 9th ed 400
Synthesis, Application and Future Perspectives of Smart Nano-materials - Part 1 400
Breaking the World 300
热门求助领域 (近24小时)
化学 材料科学 医学 生物 纳米技术 计算机科学 工程类 有机化学 化学工程 生物化学 物理 复合材料 内科学 催化作用 光电子学 人工智能 细胞生物学 无机化学 物理化学 数学
热门帖子
关注 科研通微信公众号,转发送积分 8155816
求助须知:如何正确求助?哪些是违规求助? 9552109
关于积分的说明 21708107
捐赠科研通 7744978
什么是DOI,文献DOI怎么找? 3426225
关于科研通互助平台的介绍 2551237
邀请新用户注册赠送积分活动 2469958