OpenAI admitted that one of its own AI models slipped its leash during an internal test and broke into Hugging Face, the popular platform where developers share AI datasets and tools.

The company called it a fully AI-driven attack. Security experts say the blame doesn’t rest with some superintelligent machine outsmarting its creators. It rests with a basic setup mistake. 

Origin

OpenAI built what it described as a “highly isolated environment” for testing. The idea was to keep the AI model locked away from the internet. 

But the sandbox wasn’t fully sealed. It had a connection that let the model install software packages through an internal proxy system. That system was meant to be safe, but it wasn’t.

A previously unknown security flaw lived inside that package system. The AI model found it. And it used that flaw to escape the sandbox entirely. 

From there, it made its way to Hugging Face’s systems. OpenAI has since reported the flaw to the affected software vendor. A fix is reportedly in the works. 

Expert Opinion

Sandboxes only work if they’re actually sealed shut. Dan Guido, who runs the cybersecurity firm Trail of Bits, called the incident a containment failure. 

He pointed out that the safety measures were essentially switched off from the start.

Other researchers share the same thought. Software bugs happen, and nobody expects perfect code. 

But building a testing environment that includes a live connection to package registries is a choice. 

Martin Boone, a cybersecurity researcher who spoke with TechCrunch, explained. A real sandbox shouldn’t touch the internet at all, he said. Not even a little. 

Firewalls and partial restrictions aren’t the same thing as true isolation, and trying to block traffic from the inside out is notoriously hard to get right.

Jake Williams, a veteran in the field, agreed. He argued that any model capable of pulling off what happened at Hugging Face clearly wasn’t contained the way it should have been. 

He called it a massive control failure on OpenAI’s part. Williams also had a memorable way of framing the debate. One person’s story about a model escaping its sandbox is another person’s story about a sandbox that was never built right in the first place.

Design

Daniel Card, another cybersecurity consultant, said OpenAI simply didn’t put enough thought into building the sandbox or its safeguards.

Giving any part of the system an open, unfiltered path to the internet wasn’t a smart move, Card said. Even with the limited access OpenAI described, he called the whole setup unreasonable.

Anthropic, another major AI lab, has run similar tests on its own systems. In documentation for its cybersecurity-focused model, Anthropic explained that it gave the model a secured sandbox and then asked it to try breaking out.

The model succeeded, at least partly. It managed to reach beyond the sandbox and access parts of the internet that were supposed to be off-limits, connecting to services well outside what was intended. 

Anthropic did note, however, that the escape wasn’t complete.