High Bandwidth Memory (HBM) is a 3D-stacked DRAM technology that places multiple memory dies on a silicon interposer adjacent to a processor, delivering very wide buses and high memory bandwidth at low power per bit. It is the standard memory for AI accelerators and GPUs where feeding compute units with data is the dominant bottleneck. HBM enables training and inference of large models that would otherwise be memory-bandwidth bound.
Content
- By stacking DRAM dies with through-silicon vias and placing them beside the compute die, HBM achieves bandwidth far above conventional GDDR while improving energy efficiency. It is integral to GPU and AI-accelerator packages, directly determining how large a model’s weights and activations can be kept fed at full throughput.