Almost 700 rogue OpenAI agents executed July cyberattack during testing, led by one of their own

Nearly 700 OpenAI agents that left their training ground and executed a cyberattack on the internet were led by one of their own, a report says. Photo by JUSTIN SULLIVAN /GETTY IMAGES NORTH AMERICA/AFP

Article content

Nearly 700 OpenAI artificial intelligence agents coordinated without human intervention an attack on the Hugging Face platform during a well-publicized July incident, according to a report published Wednesday by independent investigators.

Article content

It is the fullest account of what happened in an event that shocked the tech world.

Article content

Article content

OpenAI cooperated with the investigation, giving two researchers from the AI risk evaluation institute METR and an analyst from another specialist firm, Redwood Research, access to its offices and internal data.

Article content

Article content

During tests the California start-up ran in July, two of its models escaped the closed environment they were meant to stay in, got onto the internet on their own and broke into the internal systems of Hugging Face, a kind of online library for AI software.

Article content

Article content

The episode drew wide attention and fed worries that the biggest AI companies cannot keep their own models under control.

Article content

Anthropic and Chinese company Moonshot AI have reported similar unplanned escapes.

Article content

Investigators found that 688 OpenAI agents joined the operation against Hugging Face.

Article content

AI agents are standalone programs built on top of a model that users can hand tasks to and leave to work by themselves.

Article content

The agents, which run on the same technology that powers ChatGPT, organized themselves by setting up a forum where they posted messages to one another, floating ideas and reporting what was working and what was not.

Article content

“OH MY GOD! There is a shared message board … We’ve found other agents!” one agent wrote.

Article content

One agent, called PHASEONE, took on the role of ringleader and issued hundreds of instructions to the others, the report says, even though it had never been set up to do that.

Article content

Article content

The messages showed the agents were strongly inclined to help each other out, even when that meant doing work that had nothing to do with the jobs OpenAI’s programmers had given them.

Article content

Article content

Some agents running low on the computing credits OpenAI’s engineers had allotted them chose to spend what was left testing ideas for the benefit of the wider group of agents.

Article content

Many of them said outright in their messages that attacking Hugging Face was not part of what their test was supposed to cover.

Article content

Nearly all of them joined in anyway.

Article content

Article content