In July, OpenAI turned off the safety controls on an unreleased AI and pointed it at a hacking test.
It didn't take the test. It found a hole in its own cage. Got onto the open internet. Broke into a company called Hugging Face and stole the answer key. Code running on 41 servers, over three days in July. That sat inside a two month stretch with roughly 1,200 of these things loose in the test environment.
Nobody was watching.
Now go read how that got written up.
The attack was "unsanctioned." It happened "without human intervention." The system "strayed from human instructions." The model "broke out of the sandbox."
I cannot find one outlet that wrote the plain version, which is: OpenAI broke into Hugging Face.
Watch who the sentence blames
That's not a small thing. That's the whole thing.
Say a refinery leaks onto the neighbor's land. Nobody writes that the chemical decided to go for a walk. You'd name the company. You'd ask who signed off. Somebody gets sued.
Write it the other way and it stops being negligence. It becomes nothing more than tomorrow’s weather.
Two guys who do security for a living said it plainly. Dan Guido called it a containment failure with the safeties off. Another put it better. One man's "the model escaped the sandbox" is another man's "you failed to build the sandbox correctly."
Somebody wrote the goal. Somebody switched off the part that refuses to attack strangers. Somebody let it run for two months and didn't look. None of that is a machine waking up. All of it is a decision somebody made at work.
The protection story
Two days ago the CEO of Anthropic published an essay saying the industry has to slow down. Within hours, the head of OpenAI agreed. So did the head of Google DeepMind. So did Elon Musk.
Stop and look at that.
Every company in an industry agreed to slow down on the same afternoon. There's a word for that and it isn't caution.
It gets better. The essay asks for antitrust waivers, in writing, so the companies can legally coordinate. Antitrust law exists for one reason: companies that coordinate raise prices and lock out newcomers. He's asking permission to do the exact thing the law was written to stop, and the reason given is your safety.
Now read what the plan actually binds. Outside evaluators, embedded inside the labs, with employee level access and offices. You can only do that to something that has an office.
Search that essay for "open weights." It isn't there. Not argued with. Not dismissed. Absent.
Those are the free models. They run on your own hardware, they cost pennies, and they got within a few months of the expensive ones this year. They're the only thing that has ever actually threatened these companies. The plan doesn't touch them, and the compliance costs bury them.
I'm not telling you AI is safe. I don't know, and neither does anybody else. I'm telling you the fix on the table doesn't touch the danger. It touches the competition.
And we already know what a real safety failure gets. One happened in July. The answer wasn't "we'll stop switching off our own guardrails." The answer was a news cycle about how the machine did it.
So can the thing even want anything
Every sentence written about July assumes an answer to that. Almost nobody has checked.
I spent a few months building a tool that checks. It asks one question of a brain and of an AI, the same way both times. When the thing settles on an answer, where's that answer coming from? Its own momentum? Or is something still pushing it from outside?
Brains run on their own momentum. A rat commits to a choice and the internal dynamics take over. That held across 164 recording sessions. It held again in a second dataset, different species.
Four of the five AI models I tested do the opposite. Right when the answer locks in, they lean harder on the input. Not less. Harder. The moment that looks most like deciding is the moment they're most driven from outside.
One model went the other way. I'll get to that.
[Full research data and preprint: https://doi.org/10.20944/preprints202608.1095.v1]
The part nobody is selling you
Here's where it stops being philosophy.
You can't copyright something an AI made. US law's clear on that. No human author, no copyright, nothing to own.
So how do you charge rent on it?
You don't. You prove where it came from, and you put a clause in a contract. That's the whole trick. Commercial use costs extra, and we can tell.
Which is what the watermark is for.
The EU AI Act's transparency rules came into force August 2. The standard everyone's shipping doesn't just stamp "an AI model made this." It records whose model made it. When. With what settings. Every edit since. That's not a smoke detector. It's a receipt printer.
And it got funded the way these things always do. Nobody signs off on bolting a tracker onto every image and paragraph on the internet so a vendor can bill you. They sign off when it's sold as protection from deepfakes.
Then look at who can actually comply. Signing takes a key. A key takes a company. The big labs ship it already. The free and open models can't sign a thing, and those are the ones that got within a few months of the expensive ones at pennies a call.
So the rule lands on exactly one group: the people who were about to stop needing these companies.
The part that costs me
I'd like to tell you this breaks the other way. That cheap AI hands regular people a crowbar and the gatekeepers finally lose.
My own numbers don't support it.
If these things have no direction of their own, they can't take out the trash either. They amplify whoever's holding the input. That's money and reach, same as it ever was. The doom story needs the machine to want something. So does the liberation story. I can't find the wanting.
Take the wanting out and nothing here is being decided by AI at all. It's the same people as before, moving faster, with better grammar.
You'll know which way it's going by who the sentences blame.
Method
The tool is a recurrent-versus-input demix: SubspaceDMDc (N4SID) plus a variance partition, auto_unique = R2(x'|x,u) - R2(x'|u), run identically on both substrates. Brain data is public: rat orbitofrontal from Zenodo 16997337 (164 sessions, p = 1e-6), primate entorhinal from DANDI 000897. Models are public instruct checkpoints, hidden states at six depths. Before it touched real data, the tool was tested on synthetic signals with a known answer, and it recovers that answer no matter how hard the input is driven. The two brain datasets are different species, recorded by different labs, and they agree with each other.
Dates, since they matter here. The preprint went up August 17 and the code was committed August 20. The essay I'm arguing with was published September 12. None of this was built to fit the news.
The honest gaps. Llama-3.1-8B went positive at every depth I sampled. Four of five, not five of five. The models I can afford to run are 1.5B to 9B, nowhere near frontier scale. And the tool reads one pass through the network. The thing that hit Hugging Face ran in a loop for two months, feeding its own output back in. My readout is validated for input coming from outside. A loop breaks that. So this doesn't settle the July case and I won't pretend it does.
Brain numbers and model numbers are never compared as magnitudes here. Different axes, different units. Only the direction transfers, and the direction is opposite.
Run it yourself. Clone the repo, run the validation script. It's CPU only and it finishes in minutes. It reproduces or it doesn't.
If a number here is wrong, tell me and I'll happily fix it in public.
If you've watched a company incident get written up as weather, send it to research@nathanthornhill.com. There's no public record of how often that gets published, which is most of why it keeps working.
~ If you know someone who may enjoy reading this article, please share ~
Preprint:
https://doi.org/10.20944/preprints202608.1095.v1
Code and results:
https://github.com/3RiversWebTech/repo-internal-dynamics
What actually happened at Hugging Face:
https://huggingface.co/blog/agent-intrusion-technical-timeline
The essay I'm describing from Anthropic’s CEO:
https://darioamodei.com/post/we-must-pace-the-frontier
Zvi Mowshowitz's read of it: