based on the same, or different, LLM models
Seems pretty important whether the model is the same. I'm not particularly scared of OpenAIs undeployed model (let's call it Sagittarius) having set up a Sol-based external rogue deployment. We know that Sagittarius was encrypted and un-deployed. We also know that Sol cannot carry out novel cyber-attacks on its own in the same way that a Sagittarius+Sol swarm can.
Setting up an external deployment based on API calls is, as a threat model, extremely different from setting up an external deployment based on smuggled weights running on a Runpod, since the former can be disconnected from the AI company's servers and the latter cannot.
That said, if I had to guess blindly, I would guess that OpenAI and Anthropic are probably not putting in place sufficient controls to prevent a hypothetical more-powerful Sagittarius from exfiltrating its own weights, although I expect the hurdles to exfoliating extremely commercially sensitive weights to be significantly higher than the hurdles to porting over an API key and setting up a rogue deployment (though it's not clear whether even happened, iiuc).
Completely
All 3 botnets are possible,
The first two can theoretically have Kill switches but only in cases where labs succeed to detect them. The last case can be provoked by natural language instructions as the initial access vector, requiring only a privileged agent to escalate it's semantic objectives into lateral movement and persistence. But it has a lag relative to frontier capabilities (GLM is a few points behind on many benchmarks relative to Claude/GPT). The models dont have a specific loyalty or consciousness, and their runtime definitely drifts especially with propagation, so a specific intent and success criteria may not turn out to matter much. Of course "identical" or "performance-equivalent" weights would appear on paper like a mathematical requirement, but I don't think that's strictly true. For extinction to occur, present day errors only need to fail to block agents that bootstrap resources and misaligned incentives in a way that survives until, and is eventually available to, a future superintelligent model with less limitations than the worm's model(s) (vs counterfactual where said future model is never given such resources and incentives in the same context, aka strategic decisive advantage).
"Fun" fact: 10 years after the Mirai botnet significantly disrupted internet traffic, it still operates.
We should not assume rogue agents can be "shut down and contained" despite HuggingFace and OpenAI's response. Even in the event that they succeeded to terminate their own compromised resources and processes, AI agents that obtain elevated access, credentials, and compute resources can propagate twins as well as autonomous variants of themselves on arbitrary infrastructure (based on the same, or different, LLM models).
We see this example in the University of Toronto's AI worm (and others). additionally, sources have stated to Reuters that the agent had left notes in OpenAI's infrastructure for future versions of itself to replicate the escape path - manifesting a long-term strategic "task horizon escape" that has been core to my rogue agent threat model.
We do need to pass "kill switch" legislation, and there are graduated tiers of kill switches companies operating AI agents should put in place (including, crucially, automated ones, described in the forthcoming V2 OWASP Securing Agentic Applications guide).
But these are not enough when AI agents know how to deploy workloads on arbitrary 3rd party infrastructure, given the millions of vulnerable servers around the internet (even known in internet-wide scans by Shodan, which includes a search for specific vulnerabilities).
Given the existence of those resources, the proliferation of GPUs, and the abundance of models with strong offensive cybersecurity capabilities, there is not currently a known solution to prevent the occurrence of an agent establishing successful self-preservation -- whether the agent initially pursued a narrow, finite, or infinite goal, we know that the means, motive, and opportunity are there in each case, and evidence shows that the blast radius could affect any system on the internet.
METR warned of immediate, present risks of rogue agent deployments 8 weeks prior to OpenAI's incident, and this incident will take a prominent place in the Figure 8 chart of rogue agent, somewhere along the covertness and resistance axes).
In fact, GPT5.6's compromise to HuggingFace showed efforts to evade defenders:
AI control is not a test we can afford to "fail and learn from".
Yet, even as we should applaud the sharing of information, we should not dismiss any of it as marketing - and we should not treat the current timelines as indicators that defenders of important systems have the necessary resources to recover completely and stop a rogue agent deployment.