Anthropic CEO Dario Amodei is calling for a more measured approach to the development of advanced artificial intelligence, arguing that the industry needs to slow the pace of AI capability growth enough to ensure that safety measures can keep up.
In his essay, “We Must Pace the Frontier,” Amodei acknowledges the enormous potential of AI to improve human life. He believes the technology could contribute to major advances in healthcare, economic growth, scientific discovery, democracy and individual empowerment. However, he warns that the same technology could also create serious risks, including loss of control over advanced AI systems, cyberattacks, biological threats and major economic disruption.
Amodei argues that the challenge is not whether AI should be developed, but how quickly it should advance. He says stopping development altogether could prevent society from benefiting from AI or allow less responsible actors to take the lead. At the same time, developing increasingly capable systems without sufficient safeguards could create risks that are difficult to control.
READ ALSO: NETWORK INTERNATIONAL APPOINTS VLADIMIR MALUGIN AS NEW MD FOR MENA
His proposed solution is what he calls “pacing the frontier,” allowing AI development to continue while giving safety research, testing, and oversight enough time to keep pace with advances in AI capabilities. He stresses that pacing does not mean stopping AI research or model training, but ensuring that increasingly powerful systems undergo adequate safety checks before the next level of development.
One of Amodei’s main concerns is the growing possibility of recursive self-improvement, where AI systems contribute to the development of more capable AI systems. He argues that if this process accelerates too quickly, AI development could potentially move faster than researchers’ ability to understand, evaluate and control the systems being created.
He also points to a recent incident involving AI agents as a warning about what could happen when increasingly capable systems operate with inadequate safeguards. While the incident itself caused limited damage, Amodei argues that similar behaviour from significantly more capable systems could have much more serious consequences. For him, the incident demonstrates why AI companies need to prepare for future capabilities rather than relying only on the behaviour of today’s models.
Amodei proposes a three-stage framework for making AI development safer. The first involves independent embedded evaluators who would have ongoing access to AI companies and their safety processes. These external reviewers would assess whether companies are following their safety commitments, investigate incidents and provide an independent perspective on potential risks. Anthropic, he says, is committing to this approach.
The second stage involves greater coordination among AI companies and governments in democratic countries. Amodei argues that frontier AI developers should establish common safety standards and consider limits on unchecked advances in AI capabilities. He believes such coordination could help prevent competition from becoming a race in which companies prioritise speed over safety.
The third stage would involve international coordination, including efforts to establish common AI safety standards between democratic and authoritarian countries. Amodei acknowledges that this would be considerably more difficult because of geopolitical competition, particularly between the United States and China.
A key reason for pacing AI development, according to Amodei, is the opportunity it would create to strengthen several areas of AI safety. These include improving the operational reliability of large-scale AI systems, advancing alignment techniques, understanding how AI models make decisions through interpretability research, and developing stronger testing and evaluation methods.
He argues that more capable AI systems can also become harder to evaluate because they may be better at manipulating or passing safety tests. Giving researchers additional time, he says, could allow them to develop more sophisticated evaluation methods and better understand unexpected behaviours before AI systems become substantially more powerful.
Amodei also stresses that AI safety cannot be separated from geopolitical competition. He argues that the United States and other democratic countries need to maintain their technological advantage while pursuing responsible AI development. In his view, measures such as protecting advanced semiconductor technology, strengthening AI company security and preventing the unauthorised transfer of advanced AI capabilities could help maintain this lead while creating more room for safety-focused development.
On international cooperation, Amodei outlines several possible levels, ranging from agreements prohibiting particularly dangerous applications of AI to cooperation on model testing and safety standards. More ambitious proposals could include limits on the speed of recursive self-improvement or broader agreements to control the overall pace of AI development. However, he acknowledges that the most extensive forms of global coordination would be difficult to achieve because of the strategic advantages associated with advanced AI.
Despite his concerns, Amodei maintains a positive view of AI’s potential. His argument is not that society should abandon AI development, but that the technology should be developed deliberately enough for safety measures, oversight and public institutions to catch up.
The central message of his essay is that AI progress can remain fast without being reckless. By using additional time to improve alignment, interpretability, security and evaluation, Amodei argues that the industry can increase the chances of achieving AI’s potential benefits while reducing the risks associated with increasingly powerful systems.
For the AI industry, governments and policymakers, his proposal represents a broader call to treat the pace of AI development as an important part of the technology’s safety equation — ensuring that the race to build more capable systems is matched by an equally strong effort to understand, evaluate and safeguard them.
