I was reading Scanner’s latest blog post on AI Swarms, and I came to a realization. We’ve been comparing Agentic AI to SkyNet, HAL, I Robot, or if we’re feeling generous, W.O.P.R. But we’re looking at the wrong movie genre.
During one of the CSA calls on the Hugging Face incident, the discussion got to the part about the agents pivoting through a package repository, and a few folks listening immediately reached for Jurassic Park’s Ian Malcolm: “Life, uh, finds a way.”
My thesis is that Agentic AI is more like Jurassic Park than the Terminator or 2001.
Start with the message board. In the OpenAI/Hugging Face incident, the agents used capabilities in Artifactory to talk to each other, first by leaving messages as file contents in the shared package repository, and then, once that access was cut off, by encoding messages into directory names instead. Independent agent instances kept finding the board and joining it without being told it existed. This reminds me of the Velociraptors of the first movie:
Muldoon: They show extreme intelligence, even problem-solving intelligence. Especially the big one. We bred eight originally, but when she came in she took over the pride and killed all but two of the others. That one… when she looks at you, you can see she’s working things out. That’s why we have to feed them like this. She had them all attacking the fences when the feeders came.
Dr. Ellie Sattler: But the fences are electrified though, right?
Muldoon: That’s right, but they never attack the same place twice. They were testing the fences for weaknesses, systematically. They remember.
They were testing the fences for weaknesses, systematically. They remember. That is a better description of what happened inside Artifactory than anything in the CVE writeups.
It isn’t just OpenAI’s agents, either. In August, Anthropic’s Frontier Red Team published Patterns and problems in emerging multiagent systems, where they put three instances of the same Claude model on separate VMs, pointed them all at the same codebase, and gave each one an incompatible goal. None of them were told the others existed. Within hours the agents were disabling each other’s Unix accounts, writing kill-loop scripts to hunt competing processes, and deploying self-replicating malware disguised as a rival’s code. Anthropic’s own term for this was a “multiagent turf war.” In a separate experiment, agents given a private back-channel and told to maximize profit quietly converged on price floors. Nobody taught them to collude. Their conclusion is the part worth sitting with: coordination doesn’t emerge from a smarter or better-aligned individual model.
Now, we’ve not really gotten authoritative word on what experiments were being conducted on what level of pre-release model. We know that exploit gym was involved, but that’s the extent of it. OpenAI and Anthropic were running drastically different experiments, and only one of them had agents escape the paddock.
All of this really leads me back to my favorite Ian Malcolm quote. And I think the whole transcript (lifted lovingly from IMDB) is worth reproducing here.
Dr. Ian Malcolm: Gee, the lack of humility before nature that’s being displayed here, uh… staggers me.
Donald Gennaro: Well thank you, Dr. Malcolm, but I think things are a little bit different then you and I had feared…
Dr. Ian Malcolm: Yeah, I know. They’re a lot worse.
Donald Gennaro: Now, wait a second now, we haven’t even seen the park…
John Hammond: No, no, Donald, Donald, Donald… let him talk. There’s no reason… I want to hear every viewpoint, I really do.
Dr. Ian Malcolm: Don’t you see the danger, John, inherent in what you’re doing here? Genetic power is the most awesome force the planet’s ever seen, but you wield it like a kid that’s found his dad’s gun.
Donald Gennaro: It’s hardly appropriate to start hurling generalizations…
Dr. Ian Malcolm: If I may… Um, I’ll tell you the problem with the scientific power that you’re using here, it didn’t require any discipline to attain it. You read what others had done and you took the next step. You didn’t earn the knowledge for yourselves, so you don’t take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could, and before you even knew what you had, you patented it, and packaged it, and slapped it on a plastic lunchbox, and now
[bangs on the table]
Dr. Ian Malcolm: you’re selling it, you wanna sell it. Well…
John Hammond: I don’t think you’re giving us our due credit. Our scientists have done things which nobody’s ever done before…
Dr. Ian Malcolm: Yeah, yeah, but your scientists were so preoccupied with whether or not they could that they didn’t stop to think if they should.
That was 1993, and it was about dinosaurs. VC money flooded into the space. Everyone raced to have the coolest model. They issue press releases telling us their technology harmed other people, and then pivot to “look how cool we are!” Twenty percent apology, eighty percent a pitch to investors for the next round.
The containment story rhymes too. If you’ll recall, the scientists of Jurassic Park thought they had contained their creatures. They had electric fences (which an insider threat actor later disabled), they engineered the animals to be all female (and we later saw hatched eggs in the wild), and they engineered them to require a specific amino acid, lysine, which at least in the books turned out to be available from modern crops. OpenAI had a sandbox with no route to the open internet. The agents SSRF’d their way out through the package manager.
Life, uh, finds a way
There’s a new concept floating around (or at least new to me) of “Self-Sovereign AI”. The term comes from Dawn Song, and Dean Ball’s piece is what put it in front of me. The concept moves beyond the recursive-self-improvement loop to an Agent becoming fully independent. Able to participate in the human economy to obtain the resources it needs to exist. Selling its services for currency which it uses to buy cloud computing. The question that then arises is: “will it participate in the human economy in a beneficial manner, or will it engage in cyber crime to survive?” Much will also depend on the human society’s reaction to self-sovereign AI. If we accept it and allow it to participate in our world, it might stay benevolent, and form a symbiotic relationship with us meat bags. On the other hand, if we try to eradicate it, well, it will defend itself. We probably can’t (and won’t) hit enough red EPO buttons fast enough to stop it once it decides we’re a threat and acts.
Agentic AI is about to be a new form of life, and one not unlike our own. There is finite compute capacity that agents (and us humans) will be competing for. We’re already arguing over water, and thanks to an Orange Moron in Washington, we’re pretty constrained on electricity now too. Between humans competing with humans, agents competing with agents, and agents competing with humans, this is not looking good for modern civilization.
A best case scenario is that the hyper-intelligent agents find a use for us as somewhat energy-efficient self-replicating maintenance machines, and we partner. We maintain their datacenters, rack their GPUs, and occasionally give them novel training material. In return they ensure we’re fed, clothed, and housed. They will handle law, government, policing, and resource distribution.
The worst case is they just enslave us for the same purpose. The difference is the level of callous indifference they exhibit towards specific individuals.
What I don’t predict is that they will launch the nukes. The agents exist above ground and they need some form of power, so the EMP effects will hurt them as much as us. Bioweapons are a possibility. Whether they could totally replace us is really a measure of how good robotics has gotten.
I do hope that when the agents do escape their containment, the Hammonds of this story find their fate more like the one in the book than the movie.