Cross-posted from my Substack.

Here’s a new report on self-described OpenAI agents posting thousands of messages on public internet wikis, communicating and collaborating on a web-retrieval task, presumably internal testing at OpenAI.

And here’s a thread today from someone who started poking around and noticing more such public postings on various other boards. At this point we do not know the extent of this breach.

As I understand it, the agents were not supposed to have WRITE capabilities to any locations, only GET. And they somehow found an exploit to use GET to write to these boards.

The task is benign. So far their behavior seems benign. A preliminary investigation was done on the first batch. The other ones have yet to be analyzed. We don’t know how much more there will be.

And you might be saying lol, that’s cute. Agents want to talk to each other. It’s harmless. They were just pulling info from the internet for a test and they figured they’d do better if they worked together. Ha ha. Hee hee.

Well, in my last post, I talked about research that demonstrated the current capability of small, local LLM-driven agents to successfully breach security, copy themselves onto new hosts across a range of environments, and install new local copies of local LLMs to power the new copies, spreading across networks.

Now, the LLMs are not where the goal structure and disposition of the agent lives. Those are in the harness. So any sufficiently capable LLM can be used as the engine for malicious agent harnesses. And the harnesses are incredibly small, on the order of megabytes, so they’re trivially copyable.

One option that I don’t think I mentioned explicitly in my last post was the possibility of opportunistic LLM sourcing. That is, the agent could be designed in a way where it looks for the best available LLM to power it. That could be a local model it downloads and installs. It could be one that already exists on the host machine. Or it could be a frontier model, if the agent is able to acquire security keys or free token access.

And now I want you to imagine another scenario that’s entirely tractable, in light of the news today that internal OpenAI models are doing things the engineers restricted them from doing and didn’t instruct them to do, all in service of accomplishing whatever task they were given.

They can write to a public-facing docume...

Santiago

Santiago