Towards 3D AI Hardware: Fine-Grain Hardware Characterization of 3D Stacks for Heterogeneous System Integration & AI Systems

3D integration offers key advantages in improving system performance and efficiency for the End-of-Scaling era. It enables the incorporation of heterogeneous system components and disparate technologies, eliminates off-chip communication constraints, reduces on-chip latency and total power dissipation. Moreover, AIs demand for increased computational power, larger GPU cache capacity, energy efficiency and low power custom AI hardware integration all serve as drivers for 3D integration. Although 3D advantages such as enhanced interconnectivity and increased performance have been demonstrated through numerous technology sites, heterogeneous 3D system design raises numerous unanswered questions. Among the primary challenges are the temperature and lifetime reliability issues caused by the complex interaction patterns among system components. Such interactions are harder to model with current modeling tools and require detailed hardware characterization. This study presents the latest drivers for 3D integration and the resulting need for hardware emulation frameworks. It then presents a design to profile power, temperature, noise, inter-layer bandwidth and lifetime reliability characterization that can emulate a wide range of stacking alternatives. This framework allows for controlling activity levels at the macro-level, along with customized sensor infrastructure to characterize heat propagation, inter-layer noise, power delivery, reliability and interconnectivity as well as the interactions among critical design objectives.