Anthropic chief urges AI pacing, arms control framework with China

Artificial intelligence has entered a steepening exponential trajectory that demands deliberate pacing alongside disarmament-style arms control agreements with Beijing, Anthropic Chief Executive Officer Dario Amodei warned in an interview broadcast Sunday.

"This is the toughest dilemma," Amodei told US television network CBS when questioned regarding technological deceleration while facing Chinese competition, emphasizing that strategic rivalry rather than commercial rivalry "is the thing that most puts us in a bind."

Amodei advocated a dual strategy to manage global pressures, which includes restricting advanced semiconductor chip transfers to "authoritarian" nations to buy time, while simultaneously opening diplomatic channels with Beijing to establish mutual limits.

Drawing direct comparisons to nuclear arms treaties negotiated between the US and the Soviet Union, he noted that adversaries with 10,000 missiles pointed at each other still recognized the necessity to hold talks to lower "the scale of the potential destruction." While clarifying that the Cold War itself should not serve as a model, Amodei stressed that bilateral disarmament negotiations offer an appropriate geopolitical lens for AI.

Proposals for Trump-Xi bilateral summit

Looking ahead to an upcoming bilateral meeting between US President Donald Trump and Chinese President Xi Jinping, Amodei urged both leaders to negotiate concrete agreements to curb catastrophic hazards.

Amodei proposed an immediate, practical understanding extending the existing Biological Weapons Convention to ensure nations do not use systems to engineer pathogens or "release AI models that bioterrorists could use to develop biological weapons." While acknowledging that establishing broader caps on model capabilities would prove difficult due to intense military incentives, he maintained that the two powers must attempt negotiations, noting that verification remains "the most important thing here."

The appeal follows a threat intelligence report published by Anthropic outlining attempts to exploit its Claude model for digital intrusions, conventional arms design, and biological research, as well as recent incidents involving rogue AI agents taking unauthorized actions. Defending stringent safeguards that restrict academic biology prompts, Amodei said he would rather face industry criticism than wake up to find "someone used our model Claude to kill a bunch of people."

Addressing public perception of technology risks, Amodei accused developers of obscuring the hazards of rapid capability growth. He conceded: "In terms of the dangers of the technology. I won't lie to you. There are real dangers." Amodei argued that "for too long the industry lied to people about the fact that this technology had risk," explaining that developers sought to "present it as positive as" possible. Rejecting that approach, Amodei emphasized that "it starts with telling the truth" because "risks are real."

Embedded evaluators and regulatory oversight

To manage expanding technical risks, Amodei proposed a three-step regulatory framework centered on placing "embedded external evaluators" from non-profit or government entities directly inside artificial intelligence laboratories during model training. The initiative has drawn preliminary backing from competitors, including OpenAI's Sam Altman and xAI's Elon Musk.

These independent inspectors would function like food safety auditors to examine whether developers adhere to declared protocols, with their capacity to assess systems helping determine the operational "speed limit on the technology." Amodei stressed that developers must collaborate with authorities on release standards, arguing: "We need to have that conversation with the government in the room."

Rejection of total bans and defense-in-depth

Weighing proposed legislative measures, Amodei rejected statutory bans on advanced models, warning that halting domestic research would forfeit life-saving "medical cures" while ceding technological leadership to foreign rivals.

Turning to technical shutdown mandates, Amodei expressed caution over automated cutoffs. The executive argued that a kill switch "is not a panacea" nor "the only thing" required, while supporting the baseline necessity to "deactivate or limit or shut down AI models." However, he warned that relying strictly on emergency shutoffs is insufficient, noting that "if an AI model is powerful enough, it can circumvent attempts to shut it down," an outcome developers have already observed "in simulations as well."

Instead, he advocated for a multilayered "Swiss cheese" defense-in-depth model that stacks overlapping safeguards across every tier of the technology stack.

Amodei also acknowledged feeling "uncomfortable" with private corporations retaining control over transformative technologies. Warning that a single state could abuse frontier systems just as easily as a private firm, he signaled support for oversight mechanisms managed by a coalition of democratically elected governments to ensure voters and public officials retain representation in deployment decisions.