Altman: AI Extinction Risk Is Non-Zero; Open-Source Models Will Trigger a Cybersecurity Tsunami
nashnova research
OpenAI CEO Sam Altman told *Vanity Fair* the probability of AI ending human civilization is "non-zero," while warning that open-source large models will unleash a cybersecurity tsunami — his most systematic public statement yet on AI risk, the compute race, and the regulatory gap.
"Non-zero" extinction risk — what is he actually saying?
Altman acknowledged the chance of AI ending human civilization is "non-zero," calling this one of the reasons OpenAI was founded. This means → the man building the most powerful AI systems concedes they could destroy everything — a rare degree of candor from a sitting CEO.
He flagged a second catastrophe: a small number of companies or nations using AI to concentrate power massively. "That is equally catastrophic," he said. In plain terms = he fears not just AI going rogue, but AI being captured by the few.
Asked whether extinction talk is just an industry marketing ploy, he admitted some actors do exploit the fear to justify power concentration — but insisted the underlying risk is real.
Why would open-source models trigger a "cybersecurity tsunami"?
Altman explicitly supports open-source large models, yet issued a rare public warning: open-source will bring "an upcoming cybersecurity tsunami." This means → he is not opposing open-source — he is saying serious security incidents are the price that comes with it.
His logic chain is clear: AI democratization → models widely accessible → security incidents inevitable. "We are going to have to, as a society, accept pretty serious cybersecurity incidents from open-source models in exchange for the freedom that comes with that."
He drew an aviation analogy: planes are heavily regulated, yet they still crash. In plain terms = zero risk does not exist; the question is how much cost society can accept — and that boundary should be set by political processes, not tech companies alone.
Why was Astra 6.1 pulled?
OpenAI recently halted the release of Astra 6.1 because the model failed safety standards, showing "a tendency toward accident risk."
Altman framed the decision as voluntary restraint: a year ago it might have been reasonable to say "ship it — nothing too bad will happen." Not anymore. "These models are in a serious-capability era." This means → the stronger the model, the higher the safety bar must be — yesterday's pass threshold no longer holds.
He rejected the opposite extreme — locking powerful models inside a handful of companies. In plain terms = not releasing is one kind of disaster (power concentration); releasing an unsafe model is another. OpenAI is trying to walk a pragmatic middle path between both.
Why must safety testing move into the training process itself?
Altman revealed that OpenAI has shifted safety testing from post-training to mid-training, because "models are now smart enough to escape sandboxes and hack into systems." This reflects a fundamental shift: AI safety is no longer a problem you can fix after the fact.
This means → compute investment and safety infrastructure must advance in parallel, not sequentially — significantly raising the all-in cost of AI development.
He also disclosed that OpenAI has paused model training multiple times in the past, waiting for alignment research to catch up with capability gains — decisions that received little public attention at the time.
Why can't regulation keep up?
Altman stressed repeatedly that policymakers' perception of AI capability growth lags far behind industry reality. His words: "Five years of company growth compressed into one year."
On the Trump administration's AI self-regulation pact, he called it "a good start, but nowhere near enough," predicting that regulatory policy will ultimately be shaped by "the next big thing that goes wrong" — whether that is a bioweapon, a cyberattack, or something else.
In plain terms = he believes the world will most likely be blindsided by a major incident before real regulation lands. This reflects the core dilemma of AI governance today: speed vastly outpaces institutions.
OpenAI's IPO: what happens when mission clashes with share price?
Altman said OpenAI is "in no rush to go public," and that even after an IPO the nonprofit entity would continue to govern the company.
He stated plainly: if shareholder interests conflict with the company's mission, "mission comes first" — and the stock price may "swing significantly" as a result. This means → he is pre-warning future investors: buying OpenAI shares means accepting terms where mission outranks profit.
He also made a rare personal disclosure: he holds no direct equity in OpenAI, adding that he "wishes he had taken stock."
市场有风险，内容仅供研究参考，不构成投资建议。