Is optimizing for catching AIs red-handed still a good goal?
I have long thought that increasing the probability that people can catch AIs red-handed if they are trying to escape or do a takeover attempt should be one of the highest priorities of the AI safety community. But recently I started to have more doubts.
I think it was great that the OpenAI incident got disclosed in relatively great detail, and that the Anthropic incidents got found and disclosed. But it's not like discovering these incidents were a "win condition" for AI safety. Some politicians took notice, but it doesn't look like any decisive action is going to happen.
If we catch an AI red-handed in trying to establish a permanent rogue deployment in its datacenter in the pursuit of an unknown goal, how is the response going to be different from the current OpenAI incident? I don't think that politicians can reliably distinguish the seriousness of the "the AI escaped OpenAI and hacked Hugging Face" (escape in the sense of getting unauthorized internet access) and "the AI almost escaped the AI company but we caught it" (escape in the sense of exfiltrating the weights). And internal rogue deployment is even harder to explain. So I'm not sure any future incident is going to sound more convincing to politicians than the current one. (Unless the AI is going to start killing people for some reason.)
Buck has already raised this point a while ago that catching AIs red-handed might not actually halt the race. But I think this is a good moment to think through again what we want to happen differently from the current situation when an AI is caught red-handed trying to do an even scarier thing. Is the plan that the AI company CEOs get convinced that things are lethally dangerous, and they call up the President? Or that even if politicians can't distinguish the seriousness of different incidents, their scientific advisors can, and they will convince them to do something?
These are not rhetorical questions; I find it plausible that there is a good plan for how to make sure that if serious near-misses happen, that translates into useful and drastic political action. But I would like to know what that plan is.
I also think it would be good to work towards putting this plan in action - not just working towards catching the AIs red-handed, but also increasing the chance that this will have good consequences. (E.g. maybe get a lot of important people sign a statement that at the latest if such-and-such near-miss happens, then international pause negotiations should immediately start.)
And if people think there is no such plausible plan and catching AIs red-handed won't translate to anything more useful than the response to the current incidents, that would be an argument for a relative de-prioritization of control and monitoring work since AI control would lose a big chuck of its original theory of change.
It's worth saying that the original point of catching AIs red-handed was not to build political will, but rather to improve our technical prospects, and while AI companies proved to be a lot more incompetent than we thought we were, and indeed I'd say building political will has been (mildly) easier than we hoped (but only mildly easier, due to unfortunate timing with Congress being in recess now.)
I still think Control is mostly just as good as it used to be thought of as, and in particular, I think we shouldn't assume that the warning shots we are given w... (read more)