3.5D Advanced Packaging Enabling Heterogenous Integration of HPC and AI Accelerators

Exponential growth in the number of parameters used to train deep neural network (DNN)/machine learning (ML) models for artificial intelligence (AI) training/ inference applications requires extensive compute resources like CPUs, GPUs, and memory, interconnected at extremely high bandwidth. Heterogeneous integration via chiplet architectures is key to enabling economically feasible growth of power efficient computing, given the slowdown in Moore’s law. In this paper, we summarize innovative advanced packaging technologies that directly enabled the heterogenous integration of multiple chiplets including CPUs, GPUs, IO die, high bandwidth memory (HBM) die, and passive components in the largest, most complex, and high power (750 W) MI300X Instinct™ accelerator package built by AMD. Three key technologies are described: direct Cu-Cu hybrid bonding, 2.5D integration on a large silicon interposer, and metal thermal interface (TIM)-based cooling solution. The resulting 3.5D packaging technology is described and package-level reliability results are presented.