Anthropic Co-founder: AI Development Cannot Rely Solely on Tech Companies' Self-Regulation

Anthropic co-founder Chris Olah openly stated at the Vatican on Monday that the development of artificial intelligence could not be left entirely in the hands of tech companies, calling on religious leaders, governments of various countries, and civil society to strengthen external oversight.

The occasion Olah attended was quite special - the release ceremony of Pope Leo I's first encyclical on the theme of artificial intelligence. He sat beside the Pope and spoke, stating that there is a "real possibility" of AI replacing human labor on an extremely large scale.

"If this situation comes to pass, supporting those who have been displaced will be a historic moral responsibility," he said.

Olah acknowledged that AI companies, including Anthropic, have long been under multiple pressures such as commercial interests, geopolitical games, and personal ambitions, which at times conflict with the overall interests of society. "Every cutting-edge AI laboratory operates within a set of incentive mechanisms and constraints, and sometimes this set of mechanisms conflicts with doing the right thing," he added, stating that even well-intentioned researchers are inevitably influenced by these forces.

This is why Olah believes that external scrutiny is indispensable.

The weight of these remarks from the co-founder of a top AI company should not be underestimated. Amidst the increasingly fierce debate on AI regulation, Olah's choice to actively call for external checks at the Vatican is, to some extent, a candid acknowledgment of the limitations of the entire industry's ability for self-restraint.