SemiAnalysis Endorses NVIDIA's vLLM Optimization, Highlighting AMD's Software Ecosystem Gap
0xBroomberg
Semiconductor research firm SemiAnalysis praised Nvidia's performance gains on the vLLM inference engine while flagging AMD's lag on select model support. This means → the decisive battleground in AI chips is shifting from hardware specs to the depth and breadth of the software ecosystem.
What did the vLLM benchmark actually reveal?
vLLM — the most widely used open-source inference engine for large language models — served as SemiAnalysis's evaluation baseline. In plain terms = whoever runs well on this engine holds the edge in real-world deployment.
On Kimi K2.5, a hundred-billion-parameter MoE model (mixture of experts — splitting one large model into specialized sub-networks that collaborate), Nvidia's GB200 NVL72 interconnects 72 GPUs via rack-scale NVLink, pushing per-GPU throughput above 12,000 tokens/s.
AMD's MI355X fell visibly short on the same test. The core issue: it cannot match the same scale of expert parallelism and rack-level interconnect. This reflects a gap in interconnect architecture, not just single-card compute.
On the software side, what has Nvidia built — and what is AMD missing?
Nvidia's Dynamo distributed inference framework deeply integrates vLLM with three optimizations purpose-built for MoE models: prefill-decode disaggregation, efficient KV-cache transfer, and dual-batch overlap. In plain terms = Nvidia isn't just selling GPUs — it ships the software that squeezes every drop of performance out of them.
AMD still relies mainly on standard vLLM. Deep optimizations for very large MoE models and wide parallelism have not kept pace.
This means → even if AMD closes the hardware-spec gap, deployment experience will lag as long as its software stack cannot deliver an out-of-the-box solution.
"Lagging on select models" — what does that really mean?
SemiAnalysis qualified the gap as "select models," indicating AMD is not behind across the board. Its MI-series GPUs are competitive in general compute — Meta's recent large-scale procurement order is evidence.
But the word "select" exposes the real weakness: incomplete software-ecosystem coverage. Enterprise buyers need every mainstream model to run reliably, not "these work, those don't."
In plain terms = the hidden cost of maintaining two software stacks to cover different models is often the decisive factor in procurement.
How deep is Nvidia's software moat, really?
Nvidia's software barrier stacks three layers: a CUDA toolchain with roughly 4 million developers and twenty years of accumulation; first-priority support for every major ML framework; and deep lock-in through optimization libraries — cuDNN, TensorRT, NCCL.
This means → the switching cost created by all three layers together far exceeds any single hardware-spec gap. Changing chips means changing the entire software toolchain — a trade-off that rarely pencils out.
AMD's leap from "competitive in parts" to "universally deployable" may demand more time in software engineering than in hardware iteration — building broad compatibility and trust across thousands of model architectures and a fragmented developer community.
Two weeks ago SemiAnalysis said CUDA was eroding — is this a contradiction?
Two weeks earlier, SemiAnalysis noted that Anthropic had assembled a multi-platform compute architecture spanning Google TPUs, Amazon Trainium, and Nvidia GPUs. Nvidia's GPU share is being slowly eroded by custom ASICs.
Yet the positive vLLM assessment shows that Nvidia's lead in deep inference-software-stack optimization has not narrowed in step with the shifting hardware landscape.
This signals a crucial dynamic: hardware market-share loosening and software-moat deepening can happen at the same time. For AMD, the latter may be the barrier that is truly hard to close quickly.
Content is for reference only, not financial advice.