Google TPU Ironwood Leads NVIDIA B200 by Up to 50% in Performance Per Dollar
nashnova research
SemiAnalysis' first third-party inference benchmark shows Google's TPUv7 Ironwood leads Nvidia's B200 by up to 50% and B300 by 96% on cost-per-token, reshaping the economics of AI inference chips.
How much cheaper — and where do the numbers land?
On the Qwen3.5 397B model benchmark, Ironwood's inference cost runs about $0.181 per million tokens, versus $0.222 for B200 and $0.276 for B300 — roughly 19% and 34% cheaper.
This means → running the same large-model inference job on Google silicon cuts the bill by a fifth to a third.
The advantage peaks in low-concurrency, high-interactivity scenarios: Ironwood produces 50.4% more tokens per dollar than B200 and 96.0% more than B300.
In plain terms = the fewer users sharing the chip and the more real-time the conversation (think a one-on-one AI assistant), the wider Ironwood's cost edge.
Does Nvidia win anywhere?
Near the 30-second median response time band, B200 can still beat Ironwood on dollar-for-dollar performance.
Nvidia GPUs support FP4 precision — a lower-precision compute format that trades fewer data bits for faster speed. Ironwood lacks native FP4 today. This means → for workloads that push precision as low as possible, Nvidia still holds a technical moat.
When GB300 NVL72 uses disaggregated serving — splitting "understanding the question" and "generating the answer" onto separate chips — it holds roughly a 30% cost-performance edge at mid-range latency. SemiAnalysis expects this gap to close once TPUv7's own disaggregated mode ships.
Software ecosystem — has Google fixed its biggest weakness?
The long-standing pain point for external developers on Google TPUs has been the software stack: models written in PyTorch had to be translated into the JAX framework first, a clunky process with poor compatibility.
The new TorchTPU approach lets developers call `.to("tpu")` directly, no cross-framework rewrite needed. Mainstream inference engines like vLLM and SGLang can reuse most of their existing upstream code.
TorchTPU is still in private beta, with open-source planned for the October PyTorch conference. This means → the real ecosystem test comes after launch — whether it lands on vLLM and SGLang's "Day 0" support list.
What makes Ironwood's hardware tick?
Each chip packs two independent compute dies. The matrix-multiply unit uses a 256×256 systolic array — a structure where compute units pass data down a pipeline like a factory line — delivering 4× the operations per cycle versus the prior generation.
HBM memory capacity is roughly 6× that of the prior-generation Trillium, with native FP8 hardware acceleration for the first time.
Optical circuit switches scale the fabric to a 9,216-chip super-Pod delivering 42.5 FP8 exaflops in aggregate.
On the kernel side, optimizing KV-cache page layout lifted throughput by 16.5% and cut median time-to-first-token by 95%.
Who is buying — and how does the market shift?
Anthropic has committed to purchasing over one million TPUs, making it Google's largest external TPU customer. By 2029 its TPU usage is projected to exceed DeepMind's own.
This reflects a shift: Google's TPU is moving from an internal chip to a commercial product, competing directly for Nvidia's customers.
Google's eighth-generation TPU splits training and inference into two separate chips for the first time: TPU 8t for training and TPU 8i (codename Boardfly) for inference. SemiAnalysis sees TPU 8i as a direct competitor to Nvidia's next-generation Rubin NVL72.
In plain terms = Google is no longer just Nvidia's customer — it is now formally a rival. Whether it truly reshapes the market hinges on the software ecosystem keeping pace.
市场有风险，内容仅供研究参考，不构成投资建议。