HEMA-CIM: A Dual-Macro Hybrid-Domain CIM Architecture for Energy-Efficient FP MAC Operations in Edge AI

As the complexity of artificial intelligence (AI) tasks like large language models (LLMs) grows, the demand for energy-efficient floating-point (FP) computation has become critical. While the existing computing-in-memory (CIM) designs demonstrate excellent performance in inference tasks with low precision requirements, they face challenges in achieving energy-efficient FP multiply-accumulate (MAC) operations. This paper proposes HEMA-CIM, a novel dual-macro hybrid-domain CIM architecture based on: 1) a dual-macro architecture that separates computations into a digital-exponent-CIM (DECIM) macro and an analog-mantissa-CIM (AMCIM) macro; 2) a data-aware DECIM employing high-bit prediction units (HBPUs) and conditional clock gating, which leverages the statistical distribution of exponents to reduce dynamic power without compromising accuracy; 3) a highly efficient AMCIM using novel adaptive range-focusing (ARF) analog-to-digital converters (ADCs), with a two-stage, coarse-fine quantization scheme; and 4) a dynamic range aligner (DRA) and a FP formatting unit (FFU) connected with the dual macro to realize a complete FP computation pipeline. HEMA-CIM achieved a peak energy efficiency of 37.2 TFLOPS/W for BF16 MAC operations, representing a 1.18 × to 2.11 × improvement over prior work.