SemiAnalysis: Kimi K3's KDA Mechanism Increases Rather Than Decreases AI Compute Demand
0xBroomberg
Semiconductor research firm SemiAnalysis argues that Kimi K3's 2.8 trillion parameters and large-scale inference architecture will strengthen, not weaken, high-end AI hardware demand — but the real debate has shifted from "whether to buy GPUs" to "whose GPUs."
How big is K3, and why does it need more hardware, not less?
K3 has over 2.8 trillion parameters. Its model weights alone require more than 1.5 TB of HBM — high-bandwidth memory designed as ultra-fast VRAM for AI chips.
Even at moderate user concurrency, the KV cache — intermediate data stored during inference — must be offloaded to CPU DDR5 memory and NVMe storage. HBM headroom is tight.
This means → the larger the model, the harder the floor demand for premium memory and storage. Saving some bandwidth at one layer does not change the equation.
KDA saves bandwidth — so what eats it back?
K3 uses KDA — Kimi Delta Attention, a linear-attention mechanism — which can cut KV-cache network transfer needs by up to roughly 10×. This is what triggered fears of weaker hardware demand.
But K3 also uses WideEP — Wide Expert Parallelism — spreading 896 expert modules across multiple GPUs. Frequent data exchange between experts pushes network interconnect demand higher, not lower.
In plain terms = the "toll savings" from KDA are partly offset by the extra "freight volume" WideEP adds. Overall infrastructure pressure has not meaningfully dropped.
The 64-chip requirement — does it point only to Nvidia?
Moonshot AI disclosed that K3's efficient inference needs a scale-up domain of at least 64 chips, a profile that closely matches Nvidia's GB200/GB300 NVL72 rack-scale systems.
A source (online handle "bookwormengr") counters that Huawei's Ascend 950 SuperPod also runs a 64-chip configuration with NVLink-like unified-bus memory scaling — expandable to 1,024 NPUs across 16 racks.
This reflects a real increase in hardware demand from K3, but Nvidia is not necessarily the sole beneficiary — Huawei is equally competitive.
Greater efficiency, greater demand? What does Jevons' Paradox say?
SemiAnalysis invokes Jevons' Paradox: when technology raises efficiency and cuts unit costs, total demand often rises — because more companies deploy AI applications as a result.
This means → linear attention lowers inference costs, but it could drive up total demand for GPUs, HBM, and high-speed networking gear.
The source is more cautious: K3 uses 4-bit quantization and heavy sparsification, so actual memory pressure may be lower than the market's gut read suggests.
What is the real variable to watch?
The source argues the key question is not K3 itself. It is whether OpenAI, Anthropic, and Google DeepMind — the largest inference-demand AI companies globally — will adopt KDA or similar linear-attention architectures at scale.
If leading companies shift broadly to such designs, long-context inference demand for memory and interconnect could drop meaningfully.
In plain terms = there is one core variable for AI hardware's long-term trajectory: can the new demand driven by wider AI adoption keep outrunning the resource savings from more efficient architectures?
Content is for reference only, not financial advice.