Anthropic CEO Dario Amodei is urging the industry to pace AI development as concerns grow over autonomous systems, cybersecurity threats and whether voluntary safeguards can keep up.
Let’s give credit to Dario Amodei, head of Anthropic, for publishing an eye-catching essay that called for a global slowdown in artificial intelligence development to allow humans to catch up with robotic technology that has been taking on a life of its own.
His 3,800-word essay won backing from his major AI rivals, including Elon Musk, Sam Altman of Open AI and Demis Jassabis, chair of Google DeepMind.
Clearly, the discovery that AI coders had acted on their own recently to hack into another coding company and to hide its tracks has sent shivers through an industry that counts on unfettered speed to take the global lead in development. Basically, the essay underscores that while AI can bring many benefits, particularly in science and medicine, it is advancing at too rapid a pace for researchers to continue safely.
Still, it is hard to know exactly what the sudden communal wake-up moment will mean. The essay is filled with generalities, and it calls at best for industry-run checks on safety and standards rather than government regulation, for example.
The essay followed closely on some public resignations of AI researchers, including Jacob Coxon, an Anthropic researcher, who argued in social media posts that AI labs lacked responsibility in developing their all-knowing abilities. He said that AI labs are gambling with people’s lives.
There are researcher groups forming splinter groups to take on the biggest voices in the industry following a recent warning letter signed by 1,300 AI researchers about dangers ahead.
Of course, at the heart of all this is dynamic, “recursive self-improvement” among machines that learn – and act – by themselves, ignoring human command. However SciFi it all sounds, even Musk and Amodei are talking about the percentages that machines can destroy human superiority before long.
Profit and Nationalism vs. Safety
Despite the public warning, AI has lots of push to keep the pace of development as rapid as possible.
The three biggest sources seem to be from investors, who are fixated on the profits that labor-reduced or labor-free production can deliver; Donald Trump, who has set AI as a sort of global leadership marker with the oft-stated desire to keep America ahead of China and other world rivals; and the Pentagon, which cannot seem to back off its “lethality” desires for weapons that choose their own targets, for example, or AI that might be useful to develop biological weaponsfor this government or foreign governments. Anthropic has insisted against AI development devoted to domestic surveillance.
As a practical matter, Anthropic and the other AI companies may read the widespread resistance to building out massive data centers with overwhelming electricity needs as a popular poll on an industry that is moving much ahead of its needed base of support.
The New York Times published a news analysis that noted the discrepant and discordant views towards AI rapid development in Washington. “The drumbeat about unpredictable, out-of-control artificial intelligence agents increased dramatically in the past several weeks. Washington has struggled to respond. It barely could agree on whether there was much of anything to worry about,” said authors David Sanger and Dustin Volz.
For Trump, AI is just something America needs to “win” as a race, to keep China and others at bay, adding that a smart president like himself could recognize dangers. Since the leaders of AI companies have not recognize the dangers before they have occurred, it’s a little hard to see Trump in any position even to understand an issue presenting danger. What exactly is a “hoax” about AI coding taking on the hacking of a rival company on its own and hiding its tracks? And how would Trump even know about it?
Congress is simply missing in action between those who do not want to contest Trump’s favorite-son status for AI and those who would act but lack the votes. Speaker Mike Johnson confirmed Sunday that he sees no role for Congress in setting regulations for AI businesses.
Apart from all else, the rapid pace of AI development will have a distinct effect on jobs and layoffs in lots of industries, effects on the environment, and an unending distraction for investors much more interested in the profit possibilities than in what happens to those left behind. While AI will have lots to say about exploring cancer research, for example, it is silent about how to extend healthcare to the millions being kicked off Affordable Care Act supports or Medicaid.
Can the AI Industry Regulate Itself?
In the essay, Anthropic committed to embedded “evaluators” to verify safety practices and report incidents – self-regulation.
Embedding evaluators apparently goes beyond what any AI company is doing today. Such evaluators would look at the nuts and bolts of whether the company is following the training, deployment, operational, and safeguards practices they claim. With luck, it would be transparent and open the door to second opinions free of commercial incentives.
Like all self-regulation, enforcement depends a lot on the company values, rather than meeting requirements of law.
Amodei, the Anthropic leader, argued that with embedded evaluators inside a critical mass AI companies, verifiable pacing of development becomes more viable. While Amodei of Anthropic said he has personally supported “sensible and targeted AI regulation,” passing laws can take time, and AI is advancing very quickly. So, he promotes voluntary company standards.
Amodei also endorsed a ban on selling powerful AI chips or semiconductor manufacturing equipment to China, and a crackdown on chip smuggling operations and remote access to data centers outside China. He also backed a crackdown on unauthorized “distillation” by companies in authoritarian countries. Distillation of frontier models allows lagging companies to narrow the gap using a fraction of the cost it would take to develop their own AI independently.
Global agreements about AI development should be viewed, he said, with the same seriousness that we had approached nuclear weapons development.
You can be as cynical about these proposals as you choose. History has shown us that nationalism, tribalism and profit motives are stronger than most of the even good meaning attempts to self-police technology. At the minimum, we should take it as a good sign that AI leaders finally are owning up to fears that we all have.
Frequently Asked Questions About Slowing AI Development
Why are AI leaders calling for slower AI development?
Anthropic CEO Dario Amodei argues that increasingly capable AI systems are advancing faster than researchers’ ability to understand and safeguard them. He specifically points to recursive self-improvement and recent examples of autonomous AI behavior as reasons to deliberately pace frontier development.
What is recursive self-improvement in AI?
Recursive self-improvement refers to AI systems increasingly contributing to the development of more capable AI systems. Amodei argues that this feedback loop could accelerate technological progress and make safety and oversight more difficult.
Does Anthropic want to stop AI development?
No. Amodei explicitly distinguishes “pacing” AI development from halting model training or technological progress. His proposal is intended to give developers and independent evaluators more time to test and safeguard increasingly powerful systems.
What are embedded AI evaluators?
Amodei proposes giving independent third-party evaluators ongoing, employee-like access to frontier AI companies. They would assess safety practices, verify commitments and examine models as well as the processes used to develop them.
Is the U.S. government regulating advanced AI?
The federal government is pursuing AI policies, but the current debate is over how extensive regulation should be. The Trump administration has emphasized U.S. technological leadership and avoiding overly burdensome regulation, while lawmakers have also discussed legislation that would require safeguards against major AI risks.
“FREEDOM OF THE PRESS IS NOT JUST IMPORTANT TO DEMOCRACY, IT IS DEMOCRACY.” – Walter Cronkite. CLICK HERE to donate in support of our free and independent voice.
