作者
Arpan Suravi Prasad,Andrea Belano,Davide Rossi,Francesco Conti,Luca Benini
摘要
Attention-based networks, namely Transformers and variants, are widely adopted across modern AI workloads due to their strong generalization capabilities. While execution in attention-based networks is dominated by linear operations that are efficiently accelerated by specialized matrix multiplication units, the growing diversity of nonlinear operations such as activation functions, softmax, and layer normalization has emerged as a key performance and energy bottleneck. Existing solutions mostly rely on dedicated nonlinear hardware units or software emulation, which incur high area or performance overhead. In this work, we present a fused-multiply-add (FMA)-centric approach for efficient nonlinear execution. Our approach leverages the abundant FMA units already present in GEMM accelerators to approximate a wide range of nonlinear functions, including reciprocal and inverse square root, using reconfigurable piecewise polynomial approximation (PWPA), without compromising native GEMM performance. We demonstrate the effectiveness of this approach by extending an open-source GEMM accelerator, enabling the efficient execution of softmax, a wide range of activation functions (GELU, SiLU, etc.), and layer normalization with low area overhead (10%). Implemented in TSMC 7 nm technology node and operating at 1 GHz, we achieve a peak throughput of 3.9, 3.9, and 7.8 GOp/s for softmax, layer normalization, and activation functions; corresponding energy efficiencies of 92, 98, and 172 GOp/J. Our flexible, low-overhead approach improves area efficiency by$\mathbf{1. 7} \times$and energy efficiency by$\mathbf{1. 4 6} \times$for softmax compared to existing work.