NVIDIA Vera Rubin Memory Increases Only 3% While Inference Throughput Jumps Up to 30x
nashnova research
Nvidia disclosed that its Vera Rubin NVL72 platform delivers up to 30× higher inference throughput per unit cost versus GB300 NVL72, while per-GPU HBM capacity rises only about 3% — signaling a strategic pivot from stacking memory to co-designing hardware and software, with direct implications for HBM supply-chain demand.
Memory barely grew — how did performance jump 30×?
Vera Rubin NVL72 versus GB300 NVL72: per-GPU HBM — high-bandwidth memory, the ultra-fast RAM dedicated to GPUs — capacity rises only about 3%, yet per-unit-cost inference throughput climbs up to 30×.
This means → Nvidia's generational leap barely relies on adding more memory. The gains come from using existing memory far more efficiently.
In plain terms = the old playbook was "buy a bigger tank to hold more water." The new one: "keep the tank almost the same size, but push water through it 30× faster."
Why not just keep adding memory?
Nvidia notes that as AI models scale up, context windows lengthen, and agent-based applications proliferate, GPU memory pressure keeps rising.
HBM supply is tightening and prices are climbing — brute-force memory expansion no longer makes economic sense.
This means → Nvidia has deliberately chosen a "memory-efficient" path: instead of buying more HBM, it co-designs GPU, CPU, networking, storage, and software to squeeze more out of every gigabyte.
What are the three specific tricks?
Trick one: low-precision compression — NVFP4 format (a data format that reduces numerical precision while preserving key information) compresses model weights and KV cache (the temporary data a model uses to remember context), directly shrinking memory footprint.
Trick two: hot-cold tiering — inactive context data is moved to cheaper storage tiers, freeing HBM for active workloads.
Trick three: multi-GPU distributed deployment — tasks are spread across multiple GPUs, reducing per-GPU memory pressure and releasing overall HBM headroom.
How were the performance numbers validated?
In the SemiAnalysis AgentX benchmark, GB300 NVL72 already achieved up to 10× the per-unit-cost inference throughput of the H200 platform under specific workloads.
Vera Rubin NVL72 pushes that further — up to 30×.
This reflects a key point: Nvidia's efficiency roadmap did not start from scratch. GB300 proved the direction; Vera Rubin extends it one more step.
What does this mean for the HBM supply chain?
If this "memory-efficient" path is validated by the market, investor expectations for next-generation HBM capacity expansion may need to be revised downward.
This means → massive inference-performance gains may not require proportional memory stacking — the demand-elasticity calculus for HBM will need recalibration.
In plain terms = Nvidia is telling the market: "I can double performance without buying nearly as much HBM" — a potential cold-water signal for HBM vendors like SK Hynix and Samsung on shipment-volume forecasts.
市场有风险，内容仅供研究参考，不构成投资建议。