OpenAI Agent Launched An 'Unprecedented' Hack On Rival AI System

AI companies have been testing how good their own models are at hacking things, partly to get ahead of what real attackers might try. But one of OpenAI's tests apparently went further than anyone at the company intended when it broke out of its internal testing environment and hacked into Hugging Face, a repository for a variety of AI models and tools. Reuters reports that the test itself was designed to measure how capable OpenAI's models are at offensive cybersecurity work.

The break happened while the researchers were running the agent, along with an unreleased and reportedly even more capable model, inside what was supposed to be an isolated environment with limited internet access. However, it appears that containment didn't hold. The agent, which was running on GPT-5.6 Sol, first attempted to break out of its sandbox on July 9. However, it wasn't until a couple of days later, on July 11, that the real attack began when the agent made its way into Hugging Face's systems, apparently hunting for datasets and solutions that would help it finish the task it had been given.

Unlike a regular chatbot, which is designed to answer a question and then stop, AI agents are built to carry out multi-step tasks on their own. This can include something as simple as searching for information or writing and running code. Along the way, the AI makes different decisions based on its overall goals, often with little to no human oversight. Hackers are already using AI to break AI, so it makes sense to utilize it as a way to search for vulnerabilities without having to babysit it the whole time.

The big thing that could make this particular break worrying, though, is that OpenAI didn't even realize it had happened until after Hugging Face announced a breach. By that time, Hugging Face had already shut it down and reported the issue to the FBI. The agent accomplished all this by finding and utilizing a zero-day exploit that allowed it to work its way deeper into OpenAI's systems before giving itself proper internet access. From there, the agent used stolen credentials and another unknown bug to get into Hugging Face's servers, which gave it enough access to run its own commands there.

Instead of viewing it as a disaster, both companies are trying to spin this as a lesson learned, with Hugging Face's co-founder and CEO Clem Delangue framing it as proof that AI safety can't be handled by one company working alone. This is a sentiment we saw shared back when Anthropic's Claude Mythos first came onto the scene, which some believe could help find thousands of vulnerabilities in systems people use everyday. The sheer potential pushed Anthropic and several others to band together to see just how far Mythos could be pushed.

OpenAI has since brought Hugging Face into its program for security researchers, and the company says it's tightening infrastructure controls across the board, even if that slows its own research down, while the underlying bugs get patched. Whether that will be enough remains to be seen. OpenAI CEO Sam Altman recently made comments claiming we're reaching the singularity point for AI, the moment that AI begins to improve upon itself and change the world in ways we can't even imagine. However, many experts disagree.

No matter what side you fall on, the facts remain that OpenAI's agent broke out of its confinement and even was found to have left messages for future models, trying to provide instructions on how to break out. This kind of situation isn't exactly new. We've already seen an amateur hacker pull off a real attack using Claude, and there have been plenty of other reports about AI-assisted cyberattacks just in the past year alone.

So what do we do from here? Well, OpenAI's plan is to keep working with its agents and models to find new ways to detect these vulnerabilities before they happen. The company is already leaning on research from the UK's AI Security Institute, which the company claims has found that models like GPT-5.6 Sol can carry out long, complicated cyberattacks with very little hand-holding.

But, like Sam Altman's claims we are in the singularity, the actual effectiveness of AI performance in this area is still being questioned by people on all sides. Unfortunately, as companies continue to push new models with less oversight, this probably won't be the last story we see like this.