现场可编程门阵列
计算机科学
加速
内存占用
变压器
计算
硬件加速
门阵列
计算机硬件
嵌入式系统
并行计算
计算机工程
计算机体系结构
算法
工程类
电气工程
电压
操作系统
作者
Panjie Qi,Yuhong Song,Hongwu Peng,Shao-Yi Huang,Qingfeng Zhuge,Edwin H.-M. Sha
标识
DOI:10.1145/3453688.3461739
摘要
Recently, Transformers gradually gain popularity and perform outstanding for many Natural Language Processing (NLP) tasks. However, Transformers suffer from heavy computation and memory footprint, making it difficult to deploy on embedded devices. The field-programmable gate array (FPGA) is widely used to accelerate deep learning algorithms for its advantages. However, the trained Transformer models are too large to accommodate to an FPGA fabric. To accommodate Transformer onto FPGA and achieve efficient execution, we propose an acceleration framework coupling the balanced model compression at the algorithm level and FPGA-implementation optimization at the hardware level. At algorithm level, we adopt a block-balanced pruning and propose an efficient sparse matrix storage format for this pruning technique, named Compressed Block Row (CBR). At the hardware level, we design an accelerator for sparse model. And we also abstract a performance analytic model to evaluate the performance of accelerator. Experiments show that our CBR format perform better than general formats and can significantly save storage space. And our accelerator can achieve $38\times$ and $1.93\times$ speedup compared to other works on CPU and GPU respectively.
科研通智能强力驱动
Strongly Powered by AbleSci AI