Anthropic names Accenture as first embedded evaluator

Anthropic announced Friday that it has chosen Accenture and Accenture’s AI unit Faculty as its first embedded evaluator to test and assess its large models.

The work will include red-teaming, alignment assessments and tests of model safeguards.

The partnership follows a three-step proposal published by CEO Dario Amodei on Sept. 12 that calls for slowing rapid AI development to allow time to add safety measures. Amodei wrote that AI’s growing ability to build the next generation of AI, a dynamic he called recursive self-improvement, could outpace humans’ ability to understand and control these systems.

OpenAI CEO Sam Altman and SpaceX CEO Elon Musk expressed support for the proposal. Nvidia CEO Jensen Huang argued regulation was not necessary.

Anthropic and Accenture expect to commit at least $1 billion each to the effort over the next five years. Anthropic will cover Accenture’s initial work directly while longer-term funding options, such as pooled industry contributions or government support, are discussed.

Anthropic described the embedded evaluator role as new and noted that operational details are still being developed. The partnership is non-exclusive, and Anthropic plans to name additional independent evaluators in the coming weeks.

Embedded evaluators receive close access to a developer’s models and internal processes so they can perform red-teaming, alignment checks and safety testing that external audits using only public interfaces cannot achieve.

Accenture’s Faculty unit has tested models for several large AI labs and built systems designed with safety and ethical considerations. Julie Sweet, Accenture’s chair and CEO, described embedded evaluation as an emerging area and expressed readiness to partner with Anthropic to accelerate these capabilities.

Anthropic had previously committed to independent evaluation and moved quickly to select its first partner. The companies will pilot the arrangement and refine how embedded evaluation works in practice.