This is a special post for quick takes by Johan David Bonilla. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I take the loss-of-control fear seriously. But I don't think we should pause AI development, mainly because we can't afford to let bad actors take advantage of it.
A pause binds mainly the people who honor it. It slows the careful and frees the reckless. It's a bit like how countries don't want to spend billions on defense every year, but they can't afford not to. They can't stop building better weapons, because the other side won't stop either.
And even without a pause, there are already people out there building harmful AI, and we can't stop them, just like we can't stop a motivated bad actor from building a gun or malware. We also can't completely stop AI from causing harm on its own, through recursive self improvement or emergent behavior nobody asked for. Not any one model in particular, but AI as a whole.
So one of our few real chances is widespread good AI simply outnumbering bad. More good models, more capable safe labs, and more standards and regulations that push the whole field toward models that behave safely by default.
I do get the catch, though. Increased model capability arms both sides, and outnumbering only wins if defense beats offense. One bad AI could cause catastrophic, asymmetric harm that a thousand good models may not be able to stop. It kind of reminds me of nukes, in the sense that a single global superpower could technically turn the lights off for all of us.
I don't have a clean answer to the asymmetric case, or any answers for that matter. But I still don't think we should pause.
The main problem with your argument is that we may be unable to build any good AIs; you seem to acknowledge this with your statement about loss of control.
The guns analogy also seems to point in the opposite direction of your claim (for example, the UK has far fewer gun deaths than the U.S., and far less violent crime across the board. despite the fact that firearm restrictions are presumably more likely to be obeyed by law abiding citizens).