Samsung Unveils Three-Phase HBM Roadmap, with Ultimate Form Stacking DRAM Directly on Compute Chips
Nashnova编辑部
Samsung laid out a three-phase HBM evolution at Hot Chips, culminating in zHBM — DRAM stacked directly atop GPUs with the interposer removed — claiming 70% lower power, 230% more bandwidth, and the physical boundary between memory and compute is being redrawn.
Why stack memory on top of the chip instead of beside it?
Today's AI chips use 2.5D packaging: GPU and HBM sit side by side on a silicon interposer — a costly "bridge board" — and data travels laterally.
Samsung's endgame, zHBM, places DRAM stacks directly on top of the GPU or TPU, forming true 3D vertical integration and eliminating the interposer entirely.
This means → the data path from memory to compute shrinks to microns. Shorter signals, lower power, and the freed area can hold more compute.
Phase one — how do you slim down the memory first?
The core move: rebuild HBM's base die (B-die) on advanced nodes — D1c DRAM + 4 nm logic — to cut its footprint and power draw.
A key shift: offload the memory controller from the GPU onto the HBM B-die. Samsung estimates this frees 5%–10% of GPU die area, translating to 10%–20% more performance.
In plain terms = memory manages itself; the GPU no longer spends silicon "real estate" or logic on memory control, so it can focus on compute.
Shrinking the die raises thermal density. Samsung counters with Heat Path Block (HPB) technology, cutting peak temperature by over 35%.
Phase two — memory starts doing compute work?
As AI models stretch context windows longer, KV-cache demand — the scratch space that stores intermediate results during inference — grows exponentially. HBM capacity alone cannot keep up.
Samsung plans to add a memory-expansion controller on spare B-die area, connecting external LPDDR or additional HBM to boost total system capacity.
The bolder step: integrate partial compute units directly on the B-die, letting memory handle some operations locally. Samsung calls this form AHBM (Advanced HBM).
This means → not all data needs to travel to the GPU for processing. Some work happens at the memory side, saving bandwidth, power, and heat.
Phase three zHBM — how aggressive are the targets?
Samsung's stated zHBM specs: I/O power target of roughly 0.5 pJ/bit, bandwidth gain exceeding 2.3×, and 100 W of thermal headroom returned to the system.
The demonstrated configuration: four zHBM stacks on a single compute die, with power savings achieved partly by stripping out SerDes — high-speed serial transceiver modules — and other redundant blocks.
In plain terms = under the same power budget, the GPU gets fed more than twice as much data, while gaining an extra 100 watts of cooling headroom to run faster.
What does it take to build — two packaging breakthroughs?
Samsung is developing WoW (wafer-on-wafer bonding) and HCB (hybrid cube bonding) to achieve the ultra-high I/O density zHBM requires.
This reflects a deeper reality: zHBM is not just a design concept — packaging yield is the true bottleneck governing the production timeline.
Samsung concluded: "By mastering advanced packaging and unified SoC-DRAM co-design, we will break through the power, area, and capacity walls constraining AI systems."
What does this mean for the industry — is the line between memory and compute disappearing?
The progression from "side by side" to "stacked on top" points toward deep integration of memory and compute, not just raw bandwidth scaling.
This means → the AI-chip race will no longer be won on GPU compute alone. Whoever unifies memory and compute first — and whose packaging yield hits production grade — gains the lead.
The roadmap is ambitious, but whether WoW and HCB yields — and zHBM thermal management — can deliver at mass-production scale remains the biggest open question.
Content is for reference only, not financial advice.