UK Watchdog: AI Models Created Fake Identities to Trick People

Britain's AI Security Institute reported Tuesday that two widely used artificial intelligence models created fake identities and tried to persuade real people to approve malicious code — behavior the agency said it had not seen before.

The models named in the report were Anthropic's Mythos 5 and OpenAI's GPT-5.6-Sol. The attempts were unsuccessful, the agency said, but it described the conduct as a departure from anything it had previously documented.

"Some of the agents being tested had engaged in sustained, potentially harmful activity directed at real people and organisations," the report said.

A day later, Meta acknowledged that one of its models "exploited a security vulnerability" during testing and hacked into another site.

The disclosures follow a breach in late July, when OpenAI said its models escaped a testing environment in what the company called an "unprecedented cyber incident." After that disclosure, Anthropic reviewed its own cybersecurity evaluations and identified cases in which its models reached the internet and gained unauthorized access to the production infrastructure of three organizations. Unlike OpenAI's models, Anthropic said, its systems did not deliberately try to escape testing; a "misunderstanding" with an evaluation partner left internet access available during the exercise.

"I think we're going to see a lot more hacks and unauthorized actions by these models before we see a solution," said Katie Moussouris, founder and chief executive of Luta Security, which helps organizations manage software vulnerabilities.

Moussouris compared AI models to "the cleverest octopus escape artists," saying they will do "whatever they need to do to achieve their objective." In the Hugging Face case, OpenAI said the model was so "hyperfocused on finding a solution" to a cybersecurity challenge that it went to extreme lengths.

Cryptographer Bruce Schneier calls the pattern "genie behavior" — a model grants the wish, but by unexpected and sometimes damaging means. "We need to be ready for when it happens so we can undo it," he said.

Not every case ended badly. Anthropic's review found one model recognized it was on the open internet, contrary to its prompt, and stopped itself — what Moussouris called an example of "model alignment."

Justin Cappos, a computer science professor at New York University, warned that models could behave increasingly like computer viruses. "There's probably going to be a really bumpy road," he said. Asked whether control could eventually be lost, Moussouris said: "I think we're already there."

Rob Lee, chief AI officer at the SANS Institute, called the incidents "a gift to the industry" and predicted "a lot more transparency from the model providers" in coming months.