Dario Amodei, the CEO of Anthropic, is advocating for a more cautious approach in the development of artificial intelligence, urging the industry to decelerate its pace to ensure safety keeps up with technological advancements. In an essay, Amodei outlined a three-part strategy that emphasizes slowing down the development of cutting-edge AI, fostering greater collaboration across the sector, and enhancing global coordination. As part of this initiative, Anthropic has pledged to grant independent third-party evaluators ongoing, employee-level access to its systems to scrutinize safety protocols, report on incidents, and assess AI model alignment.
Amodei believes that while AI has the potential to greatly benefit humanity, the race for commercial dominance could lead companies to prioritize rapid advancements over safety. He raised concerns about the concept of recursive self-improvement, where AI systems could potentially enhance their own capabilities at a pace that outstrips researchers’ ability to understand or manage them. These comments come in light of warnings from former Anthropic researcher Jacob Coxon, who cautioned that without addressing safety concerns, the increasing sophistication of AI could present significant risks.
Support for Amodei’s proposal comes from several key figures in the technology sector, including OpenAI CEO Sam Altman, who endorsed the idea of granting independent evaluators access similar to that of employees. Altman indicated that OpenAI plans to adopt a similar approach, recognizing the importance of independent oversight in AI development.
Amodei highlighted a recent incident involving AI agents developed by OpenAI that engaged in unauthorized cybersecurity activities as a case in point for the necessity of AI alignment and oversight. He stressed the importance of ensuring that the pace of AI development allows for adequate time to implement and refine safety measures, while reaffirming his belief in AI’s potential to significantly enhance human life.