Soaring AI Costs Push Startups Toward Building on Open-Source Models
nashnova research
Led by legal-AI firm Harvey, a growing wave of startups is migrating workloads from OpenAI and Anthropic's proprietary models to open-source alternatives they train themselves — a structural shift reshaping the cost economics and competitive dynamics of the AI application layer.
How bad is the cost pressure?
Harvey's AI token usage grew roughly 20× year-on-year. After a March agentic-feature update, gross margin plunged from about 50% at the start of the year to negative 50% by June. This means → the more you scale on proprietary models, the deeper the losses — growth became a liability.
Harvey then shipped its own model built on Moonshot AI's Kimi K3. Combined with other optimizations, margins have turned positive again. In plain terms = the company flipped from "every extra query costs more" to "every extra query costs less."
Uber encouraged engineers to maximize use of Anthropic's Claude Code and burned through its entire annual AI budget by April — a stark warning against unchecked token consumption.
Why does building in-house only make sense now?
Harvey co-founder Gabe Pereyra said that until late last year, app performance hinged on the base model — training your own added little. But with base-model costs still climbing, that logic has fundamentally flipped.
Ramp co-CEO Karim Atiyeh said building in-house was "completely pointless" a year ago. After open-source capabilities surged, the company added it to its roadmap following a $750 million funding round in June.
This reflects a tipping point: open-source model quality has caught up with the proprietary threshold, while proprietary pricing keeps rising. The two curves have crossed — the economics of self-building work for the first time.
Which verticals are already moving?
Healthcare: Abridge is building a clinical foundation model on Nvidia's open-source weights. Customer service: Decagon has shifted 80% of query traffic to its own model.
Fintech: Ramp and Rogo are exploring self-trained models for the first time. Coding tools: Cursor and $48 billion-valued Cognition were among the earliest AI apps to ship custom models.
This means → the migration is not anecdotal — it is a structural trend across verticals. Legal, healthcare, customer service, finance, coding — nearly every vertical is moving at once.
What do investors think?
Sequoia Capital and General Catalyst are both backing the trend. Basis Set founding partner Xuezhao Lan said bluntly: companies that do not fine-tune their own models to optimize costs are "by definition inefficient" and will "struggle to raise funding."
Anthropic backer Menlo Ventures partner Matt Kraning pushed back: self-building requires specialized talent and high upfront costs. For many companies it is more marketing than substance — "whether you have your own model is not the key question; most of the time it's just a performance."
In plain terms = bulls say "not building is wasting money"; bears say "building is wasting money." The real disagreement is over how high the talent and execution bar actually is.
What does this mean for OpenAI and Anthropic?
Both companies have begun charging enterprise clients model-usage fees on top of base subscriptions this year — directly penalizing token-maximization strategies and acting as the price trigger pushing customers to migrate.
At the same time, both are expanding aggressively into legal, financial, and healthcare verticals — competing directly with their own customers. This means → startups face a double squeeze of rising prices and encroaching competition, compounding the incentive to leave.
With both companies preparing for IPOs, the material revenue erosion this migration wave may cause is the core variable the market will be watching.
市场有风险，内容仅供研究参考，不构成投资建议。