Avi Brach-Neufeld

I was recently rereading Option 3 from AI Futures’ excellent article How to pace the US frontier: Limit capabilities of models used for AI R&D. In this section they propose a few different options, but their primary proposal is “AI R&D can only be conducted or assisted by AIs that were trained at least 9 months ago.” This proposal has some key benefits: > 1. It delays the point at which AIs have the capability to sabotage AI research and align the next model to themselves. > 2. It likely introduces a “negative internal-public gap”, meaning that models are deployed to the public before being used for internal AI R&D. This is good for societal response. > 3. It has larger effects in worlds where takeoff is fast and therefore more risky. This also means that those who are skeptical of rapid recursive self-improvement may be open to this policy, because on their worldview it wouldn’t have as much of an effect. However, I have two major concerns with the proposal. First, it would cause capabilities to build up without real world feedback on alignment. Currently, models are trained and deployed roughly sequentially. When issues appear in a currently deployed model, the next model’s training can be modified to address it. Under this proposal, each successive model would be created without the previous model ever being deployed for AI R&D internally (and likely never having been deployed at all as labs hold back their best models to avoid accelerating competitors). Second, it would be quite difficult to determine where to draw the line as to what constitutes the end of training. The most clear-cut option would be to require that model weights remain unchanged for nine months, but this would make it prohibitively costly to fix any safety issues that arise during deployment. On the other end of the spectrum you could allow any additional training that is categorized as “safety” to be done without resetting the lock-out period. As mentioned elsewhere in the AI Futures ar