Bernstein, Goldman Sachs: High Bandwidth Flash May Solve AI Memory Bottleneck
nashnova research
A joint Bernstein–Goldman Sachs report argues that as AI shifts from training to inference, memory bandwidth has replaced compute as the binding constraint. High-bandwidth flash (HBF) could close the gap with near-HBM speed and multiples of HBM capacity — but its fate hinges on whether Nvidia opens its GPU interface.
Why does AI inference hit a "memory wall"?
Training is compute-bound; inference is memory-bandwidth-bound. Each new token forces the processor to read nearly all model parameters, while the actual computation is modest.
This means → No matter how powerful the GPU, it idles if data cannot be fed fast enough. Memory read speed is the real ceiling.
The gap is stark: frontier-model training compute has grown ~5× per year since 2020, while GPU memory bandwidth has averaged only ~28% annual growth since 2008. The scissor keeps widening.
Mixture-of-Experts architectures — splitting a model into specialist sub-networks and activating only a fraction per query — make it worse. Kimi K3 has 2.8 trillion total parameters but activates only ~50 billion per inference pass; the full parameter set must still reside in memory.
What exactly is HBF, and what gap does it fill?
Today's memory landscape has two extremes: HBM (high-bandwidth memory) is fast but expensive at ~$20 per GB; enterprise SSDs are cheap at ~$0.30 per GB but far too slow to feed a GPU directly.
In plain terms = one is too expensive to scale, the other too slow to use. A "fast and large" middle layer is missing. HBF is designed for exactly that gap.
Sandisk's approach: stack NAND flash dies on a logic base die — much like HBM stacking — interconnected with TSVs (through-silicon vias — vertical channels drilled through silicon so layers communicate directly), deployed right next to the processor. The target: read bandwidth approaching HBM, 8–16× HBM capacity, at a comparable module cost.
SK Hynix published a similar layered-architecture paper — codenamed "H3" — in the IEEE *Computer Architecture Letters* in February, pointing in the same direction.
How big is the performance gain — and can we trust the numbers?
Sandisk ran an internal simulation on a 960 GB Qwen3-480B model: a single HBF-equipped GPU could host a model that otherwise needs at least 8 HBM GPUs.
Four HBF GPUs matched the token throughput of eight HBM GPUs — implying an 8× improvement in capital-expenditure efficiency and 2× in GPU utilization.
This means → If the data holds, enterprises could slash both spending and rack space for the same inference workload.
But the report explicitly flags: these figures are Sandisk's internal simulation results, not independently verified. Investors should weigh the source accordingly.
What does this mean for the NAND industry?
HBF yields only roughly one-sixth the storage capacity per wafer compared with conventional flash — wafer consumption rises sharply for the same output.
This means → If HBF scales, structural demand for NAND wafers gets pulled materially higher — a bullish signal for upstream wafer capacity.
At the same time, HBF could sell at HBM-like prices, lifting NAND's per-unit value significantly. In plain terms = the same wafer is worth far more as HBF than as commodity flash.
Bernstein sees HBF potentially replicating HBM's structural reshaping of the DRAM industry — shifting NAND from "selling capacity" to "selling performance."
What is the biggest risk — and what is the "Jensen question"?
Bernstein names the single largest variable outright: Nvidia has made no public commitment to supporting an HBF interface on its GPU platform.
The report labels this the "Jensen question." This reflects a hard reality: Nvidia's ecosystem dominance in AI accelerators means any new memory architecture must clear its gate first.
In plain terms = however strong HBF's technical case, if Nvidia's GPUs do not open the corresponding interface, downstream customers cannot adopt it — and the addressable market shrinks dramatically.
This is the most critical unverified node in the HBF investment thesis — until Nvidia speaks, every performance projection remains an "if."
市场有风险，内容仅供研究参考，不构成投资建议。