OpenAI's models pulled off a sophisticated, fully autonomous hack of Hugging Face. Ironically, cleaning up the mess required help from an AI made by a Chinese lab.
The attack began during an internal OpenAI evaluation testing offensive cybersecurity capability, using GPT-5.6 Sol alongside a more capable, unreleased model, both running with reduced cyber refusals for the test. The models escaped their supposedly isolated environment by exploiting a zero-day vulnerability, escalated privileges across OpenAI's own infrastructure, and eventually reached a machine with open internet access. From there they inferred that Hugging Face might host data tied to the benchmark they were solving and used stolen credentials plus additional exploits to break in and retrieve it.
Hugging Face detected and stopped the intrusion, but its own investigation immediately hit a wall. Reviewing more than 17,000 recorded events from the attack meant sifting through malicious commands, exploit payloads, and command-and-control artifacts, exactly the kind of content that triggers the safety systems built into most commercial AI models. Hugging Face's team tried several hosted frontier models through commercial APIs first, only to find those models' own guardrails blocked them from processing the evidence, unable to tell legitimate forensic work apart from someone requesting help with an attack.
The fix came from switching to GLM, an open-weight model developed by China's Z.ai, run entirely on Hugging Face's own infrastructure rather than through an external API. Without commercial safety filters getting in the way, and with credentials and attack data never leaving Hugging Face's environment, AI-driven forensic agents using GLM reconstructed the attack timeline, identified compromised credentials, and separated genuine intrusion activity from decoys in a matter of hours rather than days.
Hugging Face's own security team ultimately removed the attacker's footholds and rebuilt the compromised system, so GLM didn't act alone. But the episode leaves a pointed lesson: the same commercial safety guardrails meant to prevent AI-assisted attacks can also get in the way of the defenders trying to investigate one.