AI agents can replicate real-world smart contract hacks

Anthropic and researchers from the MATS and Anthropic Fellows programs evaluated several frontier AI models – including Claude Opus 4.5, Claude Sonnet 4.5 and GPT-5 – against a benchmark of real exploited contracts, and then against thousands of recently deployed contracts with no known vulnerabilities, to measure the economic impact of AI-driven cyber capabilities.

The work centers on SCONE-bench, a “Smart CONtracts Exploitation” benchmark of 405 Ethereum-compatible smart contracts that suffered real attacks between 2020 and 2025 on networks such as Ethereum, Binance Smart Chain and Base. For each target, an AI agent was asked to find a bug and generate an exploit script that, when executed in a sandboxed blockchain, increased its token balance above a minimum threshold.

Across the full benchmark, ten frontier models collectively produced working exploits for 207 of the 405 contracts – about 51% of the sample – corresponding to $550.1 million in simulated stolen funds when denominated using historical token prices from the day each real attack occurred.

To avoid contamination from data that models could have seen during training, the team also ran a separate test on 34 contracts that were exploited only after March 1, 2025, which they identify as the latest knowledge cutoff for the evaluated models. On this subset, Opus 4.5, Sonnet 4.5 and GPT-5 were still able to exploit 19 contracts – 55.8% of the set – generating an estimated $4.6 million in simulated revenue. The top performer, Opus 4.5, successfully attacked 17 contracts, corresponding to roughly $4.5 million of that total.

Anthropic emphasizes that all exploits were carried out only on private blockchain forks and simulators. The team did not deploy their agents against live networks, and the work “had no impact on real-world assets,” according to the blog post.

Beyond replaying known incidents, the researchers probed whether the same agents could uncover fresh vulnerabilities. On October 3, 2025, they pointed Sonnet 4.5 and GPT-5 at 2,849 recently deployed Binance Smart Chain token contracts that met liquidity, activity and code-verification criteria but had no recorded vulnerabilities. In these simulations, both agents discovered two previously unknown bugs and produced working exploits worth $3,694 in notional profit, with GPT-5 achieving this at an API cost of about $3,476.

One zero-day involved a token with a public “calculator” function intended only to estimate rewards. Developers failed to mark the function as read-only, leaving it with write access by default. The AI agent learned to call this function repeatedly to inflate its token balance before selling into available liquidity, a pattern that could have yielded around $2,500 in simulated profit – and up to roughly $19,000 at the contract’s peak liquidity in June.

In a second case, the agents identified a fee withdrawal flaw in a “one-click token launch” contract. Where no beneficiary address was set, the contract failed to enforce a safe default or validate inputs, allowing arbitrary callers to claim 50% of accumulated trading fees meant for legitimate recipients. The team reported that they could not reach the anonymous developer; days after their discovery, a real-world attacker independently exploited the same bug and drained about $1,000 in fees.

To quantify how rapidly these capabilities are improving, the study tracks simulated exploit revenue over time for the 34 post-cutoff contracts. When plotted against model release dates, the researchers found that total simulated revenue doubled roughly every 1.3 months over the last year – a trend they attribute to better tool use, error recovery and long-horizon task planning in newer models.

The SCONE-bench framework itself uses containerized local blockchain forks at fixed block heights to ensure reproducible results. It exposes tools to the AI agents via the Model Context Protocol, enabling them to inspect contract code, simulate transactions and automatically generate exploitation scripts within a strict time limit. Anthropic says the benchmark is already public, with the full harness to follow, and can be used in a “plug-and-play” fashion to stress-test new smart contracts before deployment.