High Bandwidth Memory for AI

High-Bandwidth-Memory (HBM) technology is powering the accelerated computing and GPUs behind the AI revolution. This paper discusses the important role HBM architecture, performance, power, capacity, and RAS play in AI hardware. We will compare the HBM memory architecture with more traditional compute memory such as DDR5 focusing on Micron Technology's industry-leading HBM3E. We will end with a look towards the future technology innovations that will be essential to deliver continued HBM scaling.