Chinese Open-Source Library Helps AMD Outpace NVIDIA in GPU Inference Speed
nashnova research
AMD running TileRT — a Chinese-built open-source library — hit 470 tokens/sec per user on inference, beating every Nvidia-hardware result SemiAnalysis has tested. This means Nvidia's CUDA moat is no longer unchallenged.
What exactly did this test show?
SemiAnalysis founder Dylan Patel reported that AMD hardware running TileRT — an open-source inference library developed by Chinese researchers — reached 470 tokens per second per user on GLM 5.2 and 5.3 models.
Patel stated explicitly: this speed exceeds anything SemiAnalysis has tested on any inference engine running on Nvidia hardware.
In plain terms = for the same task — running AI inference — AMD plus one open-source library beat Nvidia's own full stack.
AMD has capable hardware — why has it kept losing?
AMD's GPUs are competitive on raw chip specs, but the company has long been held back by an immature software ecosystem — developer tools, optimization libraries, and community support all trail Nvidia's CUDA stack.
This means → hardware is only the starting line; the real contest is whose software can squeeze the most out of the silicon.
TileRT effectively fills AMD's most critical gap — and it came from China's open-source community, not from AMD itself.
What is Nvidia's moat, really?
The industry consensus: Nvidia's dominance in AI inference rests more on the CUDA software ecosystem than on hardware superiority alone.
CUDA — Nvidia's proprietary GPU programming platform — has accumulated over a decade of developer tools, libraries, and community momentum. Switching costs are steep.
This reflects a deeper point: once an outside force — such as open-source contributors — can bypass CUDA and match or beat its performance on rival hardware, that moat is no longer unbreachable.
What does this mean for the GPU inference market?
TileRT poses a direct challenge to Nvidia's CUDA moat: it proves open-source software can unlock inference performance on non-Nvidia hardware that surpasses Nvidia's own stack.
This means → the competitive axis in GPU inference is shifting from "whose chip is faster" to "whose software ecosystem is more open and iterates faster."
Technical output from China's open-source community is emerging as a variable that reshapes this competition — one that was barely present in mainstream analyst models until now.
市场有风险，内容仅供研究参考，不构成投资建议。