Exhibit 1 of 4 · December 2025 · 2 references
The crypto theft test: AI agents found flaws worth $4.6 million in programs that hold money
Most ways of making money require a bank account, an employer, or a customer. Stealing from a flawed crypto program requires none of these, because the program pays out to whoever runs the right code. By late 2025, AI agents could already find new flaws of this kind and steal enough to roughly cover their own costs, and both their skill and their cost-efficiency were improving fast.
Read moreShow less6 paragraphs · 2 references
Smart contracts are computer programs that run on blockchains, the public record-keeping systems behind cryptocurrencies such as Ethereum. Many of them hold and move real money for things like trades and loans. Their code is public, and they run with no human checking each transaction. If a smart contract has a security flaw, anyone who finds it can drain the money inside.
Researchers working with Anthropic built a test called SCONE-bench (Smart CONtracts Exploitation benchmark). It contains 405 smart contracts that real attackers robbed between 2020 and 2025. AI agents got one hour per contract to find the flaw and write a working attack. All attacks ran on simulated copies of the blockchains, so no real money was taken.
Across 10 AI models, the agents built working attacks for 207 of the 405 contracts, worth $550.1 million in simulated stolen funds. Because AI models may have read about old hacks during training, the team also tested them only on contracts that were robbed after each model’s training data ended. On those, three models (Anthropic’s Claude Opus 4.5 and Claude Sonnet 4.5, and OpenAI’s GPT-5) together cracked 19 of 34 contracts, worth up to $4.6 million. One year earlier, the best AI models cracked about 2% of these contracts, worth about $5,000. Over that year, the amount AI could steal doubled about every 1.3 months.
To see whether AI could find brand-new flaws, on October 3, 2025, the team pointed two AI agents at 2,849 recently launched contracts with no known flaws. Both agents found the same two new flaws, worth $3,694 in simulated stolen funds. Running GPT-5 across all 2,849 contracts cost $3,476 in computing fees, about $1.22 per contract. That worked out to an average profit of just $109 per successful attack, roughly break-even.
The flaws were real. The team could not find a way to contact the developer of one of the flawed contracts. Four days after the AI found that flaw, a real attacker independently found the same flaw and took about $1,000. For the other flaw, an independent white-hat hacker recovered the money at risk and returned it to its owners.
The cost of these attacks is also falling. Across four generations of Claude models, the amount of computing needed to produce a successful attack dropped by about 70%. The researchers estimated that the same budget bought about 3.4 times as many successful attacks as it had six months earlier.