This is a special post for quick takes by ozziegooen. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
A bunch of people in the AI safety landscape seem to argue "we need to stop AI progress, so that we can make progress on AI safety first."
One flip side to this is that I think it's incredibly easy for people to waste a ton of resources on "AI safety" at this point.
I'm not sure how much I trust most technical AI safety researchers to make important progress on AI safety now. And I trust most institutions a lot less.
I'd naively expect if any major country would throw $100 Billion on it today, the results would be highly underwhelming. I rarely trust these governments to make progress on concrete technologies with clear progress measures, and "AI Safety" is highly ambiguous and speculative.
As I've written about before, I think it's just hard to know what critical technical challenges will be bottlenecks around AI alignment, given that it's unclear when this will become an issue or what sorts of architectures we will have then.
All that said, slowing things down seems much safer to me. I assume that at [year(TAI) - 3] we'll have a decent idea of what's needed, and extending that duration seems like a safe bet.
I really want to see better strategic discussion about AI safety. If somehow we could spend $10B just to get a better idea of what to actually do, I'd easily suggest that, though strategy is something that's typically very difficult to spend money on.
Personally, this is one reason why I favor the meta approach of "make better epistemic tools, using AI." This is an area that can be very concrete and achievable, though it does have its own problems.
This is an orthogonal question. I agree that if we're there now, my claim is much less true.
I'd place fairly little probability mass on this (<10%) and believe much of the rest of the community does as well, though I realize there is a subset of the LessWrong-adjacent community that does.