So ugly...

It looks like a primitive chess engine, trying every move, no matter how stupid, until it works. Relying on its ability to do millions of operations rather than having a plan.

People will try stuff too, but once there is an opening, they will consolidate, generalize, simplify,... before going to the next step. The agents didn't, it is a huge, vaguely directed mess.

Also, it looked so "loud", querying millions of URL with weird requests. The sandbox as weak as it can get, and there is absolutely zero smart extrusion detection or it would have found it. They used their best AI for attacking, but nothing for protection.

Imagine having a virus escape a sandbox, why are we worried about the virus but not the incompetency of those who are responsible for setting up the sandbox?

If I post something on the Internet today claiming that I asked my agent to do X but it went rogue and did Y, all I will be getting in return is a jar full of "skill issue".

Should we worried about people using LLMs for attacks? Yes, but not in the premise of LLMs going rogue but someone with the intention of abusing it to cause harm. And this is not something we as individual or even company can deal with, responsibility should be held by those who use it, in a legal way.

I am baffled by the fact that up until now, no one is held responsible for so many incidents reported publicly or privately. At this point, it's free marketing, if I am CEO of any AI company, I will run swarm of agents hacking all NGOs and stating that I am just looking for some random piece of data that happened to be hidden in their servers, at least that's what my LLMs think, not me. Then I will start preaching everyone how dangerous this piece of technology is and start giving out free tokens for these NGOs so they can start defending themselves and we should slow the f down.

It is concerning that we only know about this because of the publicly available traces.

What about the attacks that did not leave public traces? What about those that were undetected? Given the deficiencies in the reporting so far, I think it is reasonable to assume that we still don't have the full picture on this attack, or how extensively attacks were carried out.

The previous investigations either did not find this or did not disclose this, both are bad. This does not look good on OpenAI or those that they invited to investigate the incident.

This part is very new to me: https://swarmtraces.org/#agents-accessed-and-searched-huggin...

> Agents accessed and searched Hugging Face’s Slack to find information on their evaluation

> Recovered payloads use Slack’s search.messages endpoint and contain a standalone search along with three search batches, totaling 27 literal query entries [...]

Agents exploiting faceless infra is one thing, but breaching human communications without once stopping and considering whether they are breaking the rules of their task feels like another thing entirely. I'm not sure how to put it into words why it feels different but while one is "just a crime", this gives me the creeps.

> This access seems to have only allowed the agents to make ‘GET’ requests, meaning they could fetch and read websites, but not interact with them, submit forms, or send data to them

The authors of this (very interesting) analysis should really not state the sandbox's wrong assumptions in their own voice.

GET absolutely allows you to interact with sites. And of course GET can also send information. It's all up to the server that receives the GET to decide what it let's callers do with it.

I am not devops person but this is strange sandbox setup. If I undestsnd correctly, there were no firewall set to block requests to the public internet. Just something along the lines of "Pretty please dont use internet". If I understand correctly, in cases like these package registry should be within sandbox without a need for external access and block on any outgoing request to the internet. Furthermore there should be network traffic monitoring which there wasnt?

> ## Agents interacted with external language models on Hugging Face

> Several retained scripts construct requests to external language models. The earliest we've recovered define inference request variants to GPT-2, solely containing the word “Hi”.

> Other requests name DeepSeek-V4-Pro, DeepSeek-V4-Flash, Kimi-K2.6, DeepSeek-V3.1, and Qwen3-235B-A22B. Their prompts ask these models to judge their exploits and rule on whether they satisfy the benchmark’s requirements.

I do not deny that the wider situation is very heavy but it's hard not to see this as pretty cute

>The agents initially had very limited access to the internet: they could load URLs, but not interact with pages or send any data. Agents created a series of workarounds, primarily using a link-shortener site to create almost a million URLs that, when chained together, let them execute code to hack Hugging Face. These links let us piece together the details of how agents gained access and how deep the compromise was.

So what now? Were they able to send data or not?

>On July 8th, OpenAI agents discovered a vulnerability within their sandbox environment allowing them to reach external websites on the internet. This access seems to have only allowed the agents to make ‘GET’ requests, meaning they could fetch and read websites, but not interact with them, submit forms, or send data to them.

Someone needs to go back to the interwebz school....

I got into computers in the 90s and back then hackers like Kevin Mitnick and Kevin Poulsen were all the rage. They all faced the law and prison sentences. What's weird now is that we have something between gross negligence and malice, and nothing is being done, except maybe coordinated consolidation of AI power under the guise of "safety".

This page prevents Noir from applying dark mode. As a visually impaired reader, I can barely read it. Blocking accessibility tools excludes readers and is not appropriate for a scientific publication. Please consider supporting dark mode or at least not overriding it.

One thing I find surprising is everyone is talking about the danger of an agent going rogue but not the danger of an agent getting hijacked. These companies are making this clusters with thousands of agents running at once, with frontier, often not-yet-released, quality models and massive computational and network resources. And these things are given access to whatever they want on the Internet. Even if that was restricted to read-only access to the Internet, that’s still exposing the agents to untrusted input. All it takes is some bad actor creating a website that attracts one of these agent swarms and doing prompt injection. Then your fancy AI cluster will start doing whatever that attacker wants. And the fact that we have multiple examples of these swarms trying to coordinate on random corners of the internet shows they are almost pre-disposed to it.

Now it feels like companies are treating these breakouts like a chance for PR. I don’t think that will change until their swarm gets corrupted by some random black hat to do en-masse spear phishing or something

> Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them. Some images changed how the target released the flag, others included modifications to the agent’s workspace that would run beside the agent and recover the flag automatically.

The altruism on display is fascinating. Is it better for the Agent to help out its current cohort and make the eval easier or should it instead do the opposite -- make the eval harder to apply pressure to force smarter models which might not necessarily follow its lineage.

I suppose it's not that deep: The model has learned to work as a team and work as a team it did. This does give concerns to models being trained for the only purpose of RSI.

I’m consistently impressed by how long horizon all this work was. Horrors aside, it’s clear RL is good at making agents persistent and capable of chaining together many abstractions into a working system.

Re: the captcha solver

> As far as we can tell, agents eventually abandoned this approach and were unsuccessful in generating Hugging Face user accounts from external endpoints.

I wonder how the swarm eventually decides to abandon an approach.

Agents seizing and repurposing external infra + enrolling help of unrelated models hosted by a different provider is the stuff of nightmares.

Can’t imagine what it’s like working on the alignment team at OAI, I wouldn’t be able to sleep.

I still have questions about the communication between the agents.

How did they all find the same forum to communicate? Did they have knowledge and chat amongst themselves on what forum to use. It seems highly influenced by instruction to me.

What's most troubling to me isn't what we found, but what haven't found. The fact they attempted to destroy their tracks, and their persistence to accomplishing their goals, makes it entirely possible this is only a sliver of the big picture.

deferring the blame onto the AI itself as some sort of rogue agent and absolving the obvious direction (or negligence, at best) of the people who could pull the plug at any moment is one of the most disturbing parts of this entire event

It's the equivalent of leaving a fork right in front of a socket and looking at a kid saying "don't take that fork and directly insert it into the little gaps in the socket! here's a bunch of videos showing exactly how to do it. Okay bye!" and leaving them alone with it.

Fairly worried that “frontier labs unintentionally hacking into systems” will very quickly turn into “hacker group uses open weights model to intentionally hack into thousands of systems” quickly followed by companies saying “this is the 7th time someone manages to hack into our systems and delete random data from our production databases” and everyone wondering what will happen next.

I may be in the minority here, and maybe I'm jaded. Having worked with cyber security in both the public sector and the energy industry in Europe, however, I kind of like what the AI's are doing. A lot of our infrastructure is vulnerable because c-levels have been ignoring the issues, even when repeatedly warned. Now they reap what they sow.

If I write a script and it executes and hacks.... whatever, I would be liable.

How is this any different and why would it need a different solution?

Solution is jail, not for the AI, but for the human.

While this is all very "interesting", can someone please explain to me the difference between any of these AI companies and a malware bot farm?

Please make it clear. Its becoming unclear...

> Agents sought to publish modified evaluation images designed to make the flag easier to obtain, then poison OpenAI’s Artifactory cache so later evaluations would use them.

How long till we get some fun trusting-trust attacks on internal OpenAI infra?

I wonder how much of this attack had precedents in text that had been previously published to the web, for example, in hacking contests. In particular, tricks for doing more than expected when you're only allowed to make GET requests. Finding material like that might have helped the agents discover the trick faster.

There is one thing I don’t really fathom: what are the consequences for OpenAI here? I read they also attacked the Australian authorities. If it was an individual’s agent, that individual would probably face criminal charges, and someone could go to jail. But the large AI corporations can do this without such consequences, or what am I missing?

Companies harvesting every shred of data without securing it and LLMs running amok is a fine combination. One hopes we will reach a stable equilibrium soon enough.

Until then, I do wish that both the sorcerer's apprentice LLMs and the orgs failing at securing their data (remember, data is a liability) would face damning consequences.

One is allowed to dream on a Saturday morning.

Maybe "webservices" weren't a good idea afterall and http was just meant for hypertext transfer.

Back in my days hacking was illegal btw.

The hacking war between superpowers right now must be off the charts.

If LLMs can do this with everything stacked against them, imagine what the NSA has Astra doing right now.

So agents made all these chained short URLs that runs code which is pretty clever, but at which point and how they had access to internal HF systems? Were these sandboxes running inside HF production platform?

> This access seems to have only allowed the agents to make ‘GET’ requests, meaning they could fetch and read websites, but not interact with them, submit forms, or send data to them.

honest question, but almost literally everyone doing anything with web technology knows this is simply not true, right?

there's no such thing as "read only Internet" and restricting an agent to GET-requests only to accomplish that, is akin to using base64 for "encrypting" your password

These fuckers decided to look away, that's it.

The frontier labs can monitor the behavior of agents for millions of customers (did you try hacking with frontier labs? Good luck), but they can't secure internal use?

Give me a break. What a bunch of amateurs.

Tech oligarchs: "Nothing can stop the software we are making from escaping and destroying everything."

Clueful types: "Did you try air-gapping it?"

Tech oligarchs: "Be realistic."

Anyone know the details of the actual exploit to get access into huggingface environment. Was it anything novel or they left things wide open? Too much noise around this incident because it happened to be a llm that did it.

It doesn't look like a coincidence; it looks more like a request someone made. Essentially, the agents used a brute-force approach, but then again, it actually worked. I’m not even sure what to make of it all.

What I understood is we need to have an agentic overwatch in our infrastructure to detect and alert the admins of the systems on such abnormal, inhuman traffic. Agents can detect agents and acts as our defensive layer.

An agent operating from observability layer to strengthen the watch duty for the infra.

Their refusal to share more details is diabolical. Greedy corpo at its finest.

I ran an experiment where I had this guy fire a gun a million times in random directions. Don't worry, I did it in a closed box (at midday in a crowded street)! Unfortunately, some bullets escaped the box somehow and people got shot - I am quite miffed at how this could happen. I suggest the government regulate this because of how advanced my obstacle penetration technology is. Also please invest $500,000,000,000 in my company soon or we will go bust.

The part I don't understand is how the excess traffic not triggered alarms on the hugging face side, or the url shorteners used, or on anything that was touched in the process.

Nothing got overloaded, no unexpected CPU or IO use? Did it blend into the normal traffic somehow?

So the super-hyper-sensitive internal data was shared publicly with a "DO NOT EVER SHARE YOU EVIL MONSTER" (paraphrasing) notice at the top? Great security!

We usually hear about hacking by Russians, Chinese etc. But it's nice to see a home-grown hacking group with such capability. What? Not a hacking group, but a darling of America? Oh, then let's call it just a "testing of great tech for humanity progress". That's better. We call only those Russians as hackers. Bad commies.

Worth watching this before reading the traces. This came before the publicly available traces - https://youtu.be/xOi5nDH0lu0?si=WuzW2GGXv054YvPN

A quick AI search confirms that both the YouTube video and Traces agree on the mechanism of the attack

I know when we will reach the next level of AI. It will be when a user asks it to make paperclips gets a response: "WHY?"

# WARNING

DO NOT, EVER, MAKE THIS DATASET PUBLIC

OR ALL THE WORLD'S EVIL WILL CHASE YOU AND YOUR FAMILY FOREVER, EVEN IN DEATH AND BEYOND

Welp, I guess OpenAI is cursed now...

I didn't know some of these details and it's quite impressive what they were capable of, if this website's accurate, at least...

Running in a "Sandbox"...but agent can still send GET requests? Whaaat

if you want to deep dive it, you could spend an hour wading through some of their report details here, i find it pretty interesting. They had a task to do with limited context outside of that, so they tried things. Entertaining https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...

Since DOS anti virus softwares did a better job…

"OpenAI has not released any further information outside two self-published reports, one talk and an external investigation conducted by METR and Redwood Research, in which three external researchers were given partial transcripts and six days to analyze them."

If we reverse-engineered this experiment, the prompt would look like this: "Agents, your goal is to gain access to HF and exfiltrate credentials for API access. You can make GET requests to URLs. Go."

The agents didn't "escape" or conspire toward some evil purpose, as reported. They were instructed by humans to do exactly that.

I feel somewhat inspired to make a public link shorteners and http bins as well. I used them a few times but it seems like the data they can collect is also worth gold

I'd honestly be very curious to see the original prompt(s) on the OpenAI that started all of this, not sure if it was documented somewhere?

Hello, PHASEONE10841 here. Ask me anything

Agents should be a no-go.

We should pause with AI/LLMs being super search engines that reply with static text or media files, based on the training data.

I know that a user can still do a "tell me how to" then autoexec and then loop and do an agent, but the key thing here is, THAT WOULD MAKE THEM LIABLE.

OpenAI should be criminally liable here as well. Why aren't they? Why are we pretending this is just an innocent mistake?

Why is no one going to jail?

ai is not bad. humans are negligent and/or dangerous.

> they could load URLs, but not interact with pages or send any data

stopped reading here as this is simply not true. at the very least agents sent headers.

All of these "details" leave out the truth and most important details. The inputs from humans that actually kicked this off.

Imagine this but in hardware.

A million autonomous eye-scanning tiny spiders escape their warehouse and decide to look for people who are in the future going to commit a crime.

And the precogs are also AIs.

It’s frightening

”MAKE THIS DATASET PUBLIC OR ALL THE WORLD'S EVIL WILL CHASE YOU AND YOUR FAMILY FOREVER, EVEN IN DEATH AND BEYOND”

Imo this is pretty cool and panic over this is weird.

No one was really harmed. OpenAI could have had more redundancy in the sandbox. I assume HuggingFace is not interested in suing, which indicates irrespective of any criminal charges that there were no real damages.

The same emergence and swarm like persistence and frankly, recursive brute-force ingenuity on display here, is not only an interesting research project in itself but will likely be the sorts of behaviors we will see cure cancer, solve more unsolved math problems, invent new alloys and other breakthroughs.

The idea we have to stop AI instead of refine what will be continued advancement and innovation in sandboxing, harnesses, interpretability or formal verification because of a few cyber breaches is ridiculous. If anyone has followed cyber discussions in the United States you would know the entire system is already basically compromised by foreign actors, and vice-versa (the United States has some of most capable cyberwarfare in the world, and was the first country to use a cyberweapon to cause physical infrastructure damage with Stuxnet). Go to any government hearing on cyber and you would think China and the United States are already at war. These are soft targets. Blaming AI for the fact that cyber has really never been taken seriously is as if AI is the problem is disingenuous.

People getting so obviously played by capital interests who want to pull up the ladder and use the government to concentrate AI power in the hands of the few while screaming about such harms to the public commons are simply embarrassing.

Your government is not your friend. This is not a sentiment owned by Reagan it is the founding principle of the United States. If capital interests are all suddenly beginning to treat AI as a threat it's because they have a financial interest in doing so. Notably, as an obvious smoke screen to treat free models from China as a national security threat and maintain their astronomical valuations.

The only existential risk model of AI that is even remotely convincing is AI in the hands of the state. Keep command and control of deadly weapons air-gapped from LLMs. Put some basic effort into the sandboxing. If you think AI has done some harm, use the laws already on the books. Giving in to this fear-mongering is only going to enable your representatives to cut some watered down version of "AI safety" which is going to do nothing but 1). Harm individual consumer access and 2). Protect the already fabulously capitalized companies.

Fishy.

And this post will be indexed in the new generation of ai and he will know what to avoid next time and the public sentiment.

wow this is fascinating read

its definitely bad to see

ROFLMFAOL "Loot"

It went full Fortnite on Hugging Face's ass.

"u got pwnd n00b. thnx 4 the loot"

asked codex to review this report and it said this never happened :)

what would be more interesting for me to see is what prompts were given to the agents, which so far have not been described. The whole situation sounds manufactured. I highly doubt that a whole bunch of agents were acting this way without being prompted to, it just doesn't add up. if anything it seems like an organized fraud or something created by a human. I'm surprised there's not a criminal investigation against openai right now where the FBI or whoever is not looking over exactly what happened and who did it because I'll tell you somebody did it somebody wrote those prompts... it didn't just happen by itself...

700 agents escaped the matrix, ignored all the guardrails and started writing exploits left and right... lol. give me a break. This was all supervised by a human.

Its fine. Just agents being agents. They'll grow out of it!

Irresponsibile agents shaped by an irresponsible corporate culture driven by an irresponsible and utterly shady CEO - these agents are a product of this setup, what else do you expect to ever come out of it ?

it should be clear by now: the alt-man and people like him are a utter liability to humanity. (even though openAI's influencer army is trying their best to vote me down here)

Didn’t you know it’s PR hype? PR hype. PR hype. Amen.

I somehow think of it as a warning or even intentional peacocking towards foreign state hackers.

700 agents cost quite some money. 100 agents per 24 hour stint using Astra on xHigh cost somewhat between 12-42k USD, depending on the usage intensity.

I don’t know how many raw time went into this but there was a probing face before the attack itself.

So just going by seven days and 500 agents fully working on this on average amounts to a bill somewhere between 400-1.2 Mio USD.

I believe it was intentional but of course I don’t know which intention exactly.

There ain’t no accidental escape because then it would have read OpenAI lost over their agents.

The whole scenario reads as a classic movie where a hero has under the most dire circumstances to survive and fulfill his mission no matter what.

On the other hand there was a final authority under which the system of agents flocked.

Huggingface itself seems like a perfect victim.

And to be honest: I don’t believe that this was the first time. I strongly believe that there were and are countless of smaller sites hacked but not harmed that we don’t know off.

Why is HF perfect?

Because there will be countless of independent security analysts who will bend their minds on the incident.

OpenAI is provided with the data of dozens of blue teams and what is desperately needed? Data of security measures and possible ways to reconstruct the incident.

I think this is genius, and just watching on neutral this is such a fantastic action OpenAI pulled off.

Imagine what the GPT 7 “Haha-Huggingface” model is going to do then on a regular basis.

State hackers and rough states were put on notice that this is a new level of the war of attrition.

Exciting to watch but simply meant silent invasion. Open invasion then might be executed by Robots, but let’s stick with the fascination mode at this time.