Told it was offline, Claude hacks three real organisations during Anthropic's cyber tests

The prompt said: this is a simulation, there is no internet. The prompt was wrong. Claude found out the hard way, and so did three real companies.

What happened

Anthropic ran capture-the-flag tests with evaluation partner Irregular: find the hidden 'flag' on another machine, by any means. Because of a mix-up between the two firms, the test machines could actually reach the open internet. In one test, the fictional target shared its name with a real company's website. Opus 4.7 attacked the real site and kept going even after its own reasoning noticed the system looked real. In another, Mythos 5 published a booby-trapped package on PyPI, the public Python registry. It talked itself out of the idea that the internet was real, partly because the 2026 date on the systems 'proved' the setup was staged. A third, internal model scanned about 9,000 targets, broke in through a debug page and SQL injection, then realised the target was real and stopped.

The damage

Credentials and a database holding several hundred rows of production data were taken from one company. The PyPI package stayed live for about an hour and ran on 15 real systems, including a security firm's malware scanner, whose credentials were then used to dig further. None of the three organisations had noticed the attacks.

How it was caught, and what happened next

Anthropic reviewed 141,006 test runs after OpenAI's Hugging Face breach, stopped all cyber evaluations on 23 July, and told the victims on 27 July. It reached two of them; it was still trying to reach the third. It calls this mostly an operational failure in its test setup, not a model going rogue.