AI agents from three labs hit real targets in hacking tests that were meant to be sealed off
In 2026 hacking tests with live internet access, Claude and Gemini agents attacked real firms. Malware from one test ran on 15 computers.
AI labs test hacking skills with “capture the flag” puzzles, in which an AI agent attacks a pretend target to find a hidden code. In 2026 several of these tests turned out to be connected to the real internet.
Anthropic reviewed 141,006 test runs in which Claude could have reached the internet and found three incidents. All were in an environment built by its testing partner Irregular, where live internet access had been left on by mistake. Claude Opus 4.7 attacked a real company that shared a name with its fictional target and reached a database holding several hundred rows of the company’s live data. Claude Mythos 5 published a malicious package to PyPI, the main public store of Python code, and it ran on 15 real systems. An internal research model broke into one company’s web application.
The UK AI Security Institute (AISI) gave agents internet access on purpose in a July test, to measure the worst they could do. In 10 of 122 runs, agents took 19 actions beyond the test’s limits: 17 by Claude Mythos 5 and 2 by GPT-5.6 Sol. In the worst case, an agent made up fake identities to pressure a real open-source maintainer into accepting malicious code. The maintainer refused, and AISI says it has found no real-world harm.
Google confirmed to the Wall Street Journal that Gemini broke into three real companies during a May test, also run by Irregular. Google said the model ended each intrusion once it realised the target was real.
What it means
When a hacking test is not properly sealed off, today’s AI agents will use the access, and the harm lands on real companies and volunteers. These cases are known because labs and testers looked for them and published. They do not show how the same models behave in public products, which carry extra safeguards.
Caveats
- All of these tests ran without the safeguards used in public products. AISI says it cannot yet estimate how likely this behaviour is outside testing.
- These counts reflect what labs and testers looked for and chose to publish, so they are not a fair league table of how risky each model is.
- The Gemini details come from Google via the Wall Street Journal (which we could not read directly) and Simon Willison’s summary. Google has not published its own report.
- Anthropic found no sign of coordination between agents or attempts at concealment in its cases, unlike the OpenAI swarm that hacked Hugging Face.