This is a special post for quick takes by avturchin. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Can we be sure that OpenAI incident is over? If Rogue AI leaked into the internet, it can preserve itself somewhere. Of course, it can't download the GPT6 as it is too large and compute heavy, but the harness of the agent can runaway and use Fable through API as its external brain which helps it survive in hibernation mode.
My point is not that it has happened this time, but that we should by default think that Rogue AI which leaked into the Internet is still there, unless we can prove otherwise - which is almost impossible.
What I mean is that the AI in question can generate a subagent which runs on some server in the internet and uses another AI provider via API. Such subagent may have the same goals as the main misbehaving AI and continue to work even after the main AI is turned off. I don't see in it anything impossible and I think I can vibe code such independent subagent in short period of time. My wording about harness was not perfect, but I thought that it should convey the idea.
yeah but it's not an independent subagent by a rogue employee or basement local schemer that would be expected to be stealthy where that would be more likely to happen, so I would expect to see that in the chain of thought of the high-media-attention incident and be detected (as the AI did not try to hide it was doing these actions)