I get the impression that most people in AI safety view it as unlikely that we get government action that significantly decreases AI existential risks in time, in worlds where something like RSI at the pace of AI 2027 is possible. I personally think it is ~70% likely that we get some sequence of timely government actions over the next 3 years that will successfully avoid most short term risks, and I don't know what the points of disagreement are.
Some things I believe that influence my optimism:
The median outcome in my head is something like: "An RSI ban becomes the democratic party line AI safety priority, and it's either included in the first continuing resolution of the new congress, or it passes with bipartisan support before that. This acts as a major stopgap on current capability improvements, and pressures Trump into trying to negotiate on international AI regulations with China. The agreement isn't completely optimal but does slow progress a lot, enough to buy time for either whoever wins the 2029 election to negotiate a stable pause, or just for public sentiment to shift such that Trump negotiates a stable pause himself."
I absolutely do not understand all the dynamics at play here and am making a ton of assumptions, but I am reasonably confident that a majority of the things I listed here will prove true. In part I am writing this for calibration.
Mostly, I worry that some people may be overindexed on the pre Coxon era of AI safety politics. My brain says that the details I'm glossing over probably contain many practical challenges which I'm not giving enough credit to. My gut says that people might be reasoning from the assumption that politics is difficult, then coming up with potential bottlenecks to support that idea. To be fair, I am absolutely doing the inverse of this! I still can't shake the feeling that the pessimism is just not justified, much danger remains but I find it more likely than not that current institutions are equipped to manage the short term.
Also, I only started to think everything would flow through an RSI ban while writing this out, all other proposals to buy time felt a lot less natural. It kinda just seems like the most legibile approach to get good safey legislation, so I think it'll probably end up as the workhorse #1 priority, though I don't think it needs to be for most of what I said to happen.
Could you talk me through how you would frame a law that banned RSI?
There are two things I think it could cover and I'm not quite sure which one you are pointing at.
The first is something like "explicit RSI" - ask some agents to work on building better AI models.
The second is more like "implicit RSI". You're working at a frontier lab, with access to frontier models, part of your job involves trying to make better models, and it is fairly mundanely ordinary that software and research tasks are assisted by AIs these days. As the models get better, they get b... (read more)