Anthropic has announced a partnership with Accenture to carry out independent evaluations of frontier AI models. The initiative supports a commitment outlined by Anthropic’s CEO in the essay “We Must Pace the Frontier,” which called for independent evaluators to be embedded within AI companies.
The partnership will be led by Faculty, Accenture’s specialist AI business, and will focus on evaluating and red-teaming AI models, conducting alignment assessments and testing model safeguards. Accenture’s experience helping businesses and governments deploy AI across different industries is expected to provide an enterprise perspective to the evaluation process.
ALSO READ: BANKIT REBRANDS AS WAYVEPAY FOR DIGITAL BANKING GROWTH
$1 Billion Investment in AI Evaluation Capacity
Anthropic and Accenture each plan to invest at least $1 billion over the next five years to build capacity for independent AI evaluation. The companies say the investment will support the development of capabilities needed to assess increasingly advanced AI systems.
Anthropic described embedded evaluation as a relatively new approach, noting that many aspects of how it will operate are still being developed. Unlike traditional external evaluators, embedded evaluators would work within AI companies and receive access similar to that of employees.
According to Anthropic, this access could enable evaluators to examine how AI companies operate, assess whether safety commitments are being followed and identify potential blind spots. Embedded evaluators could also report incidents and provide the public with additional information about the potential benefits and risks of advanced AI.
Developing Standards for Independent AI Oversight
Anthropic noted that embedded evaluation is still an emerging approach, with no established standards yet governing the information evaluators should access or how their findings should be reported. There is also no widely established long-term funding model for independent evaluation.
The company said it is also in discussions with METR and other nonprofit evaluators to pilot elements of embedded evaluation using their own funding arrangements. Anthropic said it ultimately wants to see an ecosystem of independent evaluators operating under shared standards.
The partnership with Accenture is non-exclusive, meaning Anthropic plans to work with additional evaluators, while Accenture will also work with other AI developers. Anthropic said it will continue developing frontier AI models while bringing independent evaluators into the development process and sharing lessons as the approach evolves.
