Anthropic CEO Dario Amodei on Saturday called for a slowdown in the development of artificial intelligence — a stunning admission that the rapidly accelerating technology poses unknown existential threats to humans.

In a lengthy essay published online, Amodei urged the industry to “slow the pace at which we improve the capabilities of AI models” and warned about the potential dangers of rogue AI agents.

“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” he wrote.

Amodei cited two major concerns: The technology’s dramatic advancement “driven primarily by AI’s growing ability to build the next generation of AI,” and recent cyberattacks initiated by OpenAI models on the open source software platform Hugging Face. He suggested safeguards, including third-party evaluators to ensure adherence to safety practices.

“I continue to believe that AI can enormously improve the quality of human life. My desire to achieve these benefits is undimmed,” he wrote. “But the benefits will only be achieved if we build the technology in the right way, and — so long as we use the time we gain well — it is worth taking unusually deliberate care to get it right.”

Sam Altman, the CEO of OpenAI, wrote on X that he agreed with Amodei that “we need to pace the frontier,” a topic that Altman said his company had been discussing in recent weeks.

Elon Musk, the CEO of Tesla and SpaceX, also weighed in, saying he agreed with Amodei.

Anthropic’s head of policy, Sarah Heck, followed Amodei, praising congressional leaders who have proposed competing bipartisan approaches to the regulation of AI. She said in a statement posted to social media that the government has a “critical role to play here, including blocking the sale of the most advanced chips to adversarial nations like China, enacting a national law requiring testing of frontier models, with the power to block the most advanced models that prove to be unsafe.”

Amodei’s essay follows claims from multiple AI researchers this week that the companies developing artificial intelligence are doing so knowing that it poses serious risks.

Those warnings have raised alarm among U.S. lawmakers, some of whom acknowledge that the government is ill-prepared to institute guardrails on the technology.

Rep. Ted Lieu, D-Calif., reacting to the developments on Saturday, said on X, “Multiple AI companies are now basically saying what they are creating is not safe.”

“The people building AI earnestly believe that it could kill us all by the end of the decade,” Jason Coxon, an Anthropic researcher who resigned, wrote in a viral thread on X on Thursday. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”

Anthropic’s chief safety researcher, Evan Hubinger, later responded that Coxon was right. “We really do earnestly believe AI could kill all humans,” he said. “I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

Last month, Microsoft founder Bill Gates laid out his concerns about AI in an essay, writing that the speed at which AI is advancing will destabilize the economy, empower people and other AI to “do more harm,” and exacerbate psychosocial problems.

Those urgent warnings stand in stark contrast to President Donald Trump’s public stance on AI. When asked Thursday if he had concerns about the risk that AI poses to humanity, he said, “No, I don’t have any.”

Trump added that he is focused on whether the U.S. will “win AI” against China, which is also confronting dangerous security risks from AI.

The post Top AI leaders propose slowing down AI development appeared first on MS NOW.

Author: Source Author

From MS Now.