计算机科学
计算
电子工程
推论
电气工程
数字信号处理
信号处理
电子设计自动化
数字电子学
电网
等效电路
计算机硬件
作者
Bo Liu,Xilong Kang,Weixuan Wang,Yang Zhang,Bo Hu,Xinyang Chen,Xin Cao,Xinming Yan,Yuan Song,Zihan Zou,Zhen Wang,Xin Si,Hao Cai
出处
期刊:IEEE Transactions on Circuits and Systems I-regular Papers
[Institute of Electrical and Electronics Engineers]
日期:2026-01-01
卷期号:: 1-14
标识
DOI:10.1109/tcsi.2026.3742618
摘要
Large language models (LLMs) have become a core component of modern intelligence systems. However, their considerable amount of parameters and auto-regressive decoding nature has led to a surge in data transfer and computation. Computing-in-Memory (CIM) architectures are widely accepted as a promising paradigm for accelerating LLM inference by mitigating the memory wall. Yet previous works on CIM neglect the evolution trend of Key-Value (KV) sharing mechanism. Furthermore, directly mapping this mechanism onto conventional CIM introduces mismatches that span from off-chip data movement to on-chip operation execution. To bridge the gap between KV sharing mechanism and CIM architecture, we present S2-CIM, a statistics-aware CIM accelerator that progressively exploits the intrinsic data non-uniformity within KV sharing-enabled LLMs across three levels: 1) A page-aware sparsity scheduler that densifies fragmented KV cache access patterns into group-wise compact fetches, reducing external memory access by 66.1%; 2) A pattern-aware 4-2 compressor that simplifies accumulation logic by injecting zero-expectation errors into low-probability bit patterns, saving 32.4% area and 55.7% energy; 3) A special function converter that transforms non-linear operations into CIM-native multiply-accumulate operations via curvature-proportional radix-2 interpolation, reducing system-level latency by 18.2%. Fabricated in 28nm CMOS, S2-CIM achieves a peak energy efficiency of 116.83TOPS/W and delivers an average energy cost of$172.5\mu $J/token on LLaMA3.1-8B, representing$1.31\times $–$2.57\times $energy savings over the state-of-the-art CIM accelerators with only 0.23 perplexity degradation.
科研通智能强力驱动
Strongly Powered by AbleSci AI