SemiAnalysis: Memory Shortage to Persist for Years, CPO Deployment Delayed to 2029
Taylor Wilson
SemiAnalysis founder Dylan Patel lays out updated calls across the AI infrastructure stack: memory still has 2–3× upside, and CPO mass production is now pushed to late 2028–2029 — two thesis lines that will reshape pricing from chips to consumer electronics.
How much is AI actually burning?
Anthropic turned free-cash-flow positive in Q2 this year. Both April and May were profitable, with annualized recurring revenue topping $50 billion and gross margins above 70%.
This means → the leading AI companies are no longer pure cash incinerators. Revenue scale and margins can now fund continued infrastructure buildout.
Patel used his own firm as a case study: SemiAnalysis's 90-person team went from under $100,000 in annualized AI spend last November to $11 million today, with a peak-week run rate touching $14 million. "AI costs already exceed a third of our people costs — and will likely hit half before year-end."
Why isn't memory just a normal price cycle?
Memory capacity grows 20–30% per year. AI-side demand is doubling. The gap keeps widening. Patel's call: "This is not a short-term shortage — it is a structural shortage that will last years."
The core driver is the impact of reasoning models on KV cache — the temporary memory area that stores context during inference. Models like o1 have caused context lengths to explode, inflating KV cache and making memory the most direct beneficiary.
This means → supply rigidity will force downstream users to reallocate limited memory. Consumer electronics, with low price elasticity, takes the hit first: low- to mid-tier phone makers have already seen shipments drop 40%, and iPhone and MacBook prices will rise next year. In plain terms = memory keeps climbing until AI has taken what it needs — only then does consumer electronics stabilize.
How far can the CPU catch-up run?
Reinforcement learning requires heavy CPU use for environment verification. Agentic inference leans on CPU compute. Combined with years of AI-chip shipments outpacing matched CPU supply, the industry is now in a concentrated backfill phase. Nvidia's Vera CPU already carries a $20 billion revenue guide.
Patel flags an explicit warning: "There is a large backfill effect in these numbers." Once the historical deficit is closed, only incremental demand remains.
In plain terms = a single Blackwell runs roughly $50,000; a CPU runs roughly $5,000. Even at a higher attach rate, CPU dollar volume stays far below AI accelerators. Memory and AI accelerators are the big-ticket items. CPU is a re-rating from undervaluation — now more fairly priced, but it will not outgrow AI accelerators indefinitely.
Why is CPO delayed again?
Patel explicitly pushes CPO — co-packaged optics, integrating optical communication modules directly onto the chip — to late 2028–2029 for mass production. Manufacturing yield, chip design, and supply-chain readiness all fall short of deployment-scale requirements.
Nvidia's Rubin and its successor architecture Feynman will still use an all-copper design. Downstream chip changes — such as Rubin Ultra's Kyber dropping its 800V design — further delay CPO adoption.
This means → the copper-cable window extends. Amphenol and other copper-connector makers will benefit more than expected. Patel's stance: SemiAnalysis is more bullish on copper and non-CPO optical solutions over the medium term. "CPO will happen long-term — copper will eventually be replaced — but the timeline has been pushed back."
Where does the power come from?
Patel forecasts new data-center power additions: 20 GW this year, 30 GW next year, 50 GW the year after.
He expects half of new data-center power over the coming years to come from "behind-the-meter" generation — on-site power built by the companies themselves, not drawn from the public grid. The dominant solution today is combined-cycle gas turbines from GE Vernova, Mitsubishi, and Siemens.
This reflects a deeper shift: power is no longer a support function for data centers — it is the hard constraint on expansion speed. Patel adds that within roughly two years, solar-plus-storage will undercut gas on a blended cost basis. The longer-term frontier is orbital data centers — deploying compute chips in space, where solar panels avoid atmospheric losses and energy density far exceeds ground level. SemiAnalysis's largest research division is no longer semiconductors but what it internally calls "DEI" — data centers, energy, and industrials.
Content is for reference only, not financial advice.