Yesterday we released AI 2040: Plan A, but there's lots of work left to do.

We still have tons of uncertainty about the future of AI and the best strategies for how humanity can successfully chart a path through the intelligence explosion. You can read much of our current thinking on the supplements page.

In this post, I'll outline some research areas related to the feasibility/desirability of Plan A that I am excited about further research into.

We at AI Futures Project will do some of this work ourselves, so if you’re making a serious attempt, please get in touch. (Or apply to work with us directly). 

(Note that many of the below proposals are only relevant in something like a Plan A world where we’ve slowed the intelligence explosion massively).

Write Additional Prescriptive Scenarios 

There are many other proposals for how to navigate the intelligence explosion. Some of these plans are promising, some less so. However, none of them have been publicly gamed out in scenario format in nearly as much depth as Plan A. Instead, they tend to operate at a more abstract level that attempts to be consistent with many possible futures. Unfortunately, this also makes it much more difficult to analyze how good each of these plans are, because the tradeoffs have not been explored in depth in any possible worlds, and because it’s more likely that there are hidden difficulties or contradictions in a plan if it hasn’t been gamed out concretely even once.

I would like to build out a large set of competing concrete scenarios, some predictive, and some prescriptive (i.e. laying out concrete plans). I'm especially excited about doing this for plans that are potentially competitive with Plan A, including: 

• Plan S: Indefinite halt. A halt on all frontier AI capabilities progress, intended to last at least a few years. Different variants of Plan S have different conditions for resumption of AI progress; for example it could be alignment progress, lie detectors, human uploads, or intelligence enhancement.
• Domestic-first Plan A. Regulate AI domestically enough to reduce AI takeover risk to acceptable levels, which will require a long slowdown. It’s possible other countries will also regulate domestically and something like Plan A won’t be needed; otherwise, transition to Plan A later. This is similar to what Richard gestures at in his critique of AI 2040. 
• GPU Arms Control. International agreements for countries to limit their GPU stock or flow, analogous to historical arms reduction agreements.
• CERN for AI. An international project to develop frontier AI, with all other projects regulated to be substantially behind in capabilities.

See more discussion of these options here in our scenario. 

A second class of prescriptive scenarios that would be useful are scenarios that are similar at a high level to Plan A, but with some significant perturbation(s). For example, Tom Davidson critiques Plan A for adding “dry tinder”, i.e. lots of additional compute that is being prevented from doing capabilities research on the critical path for the intelligence explosion, but if the regulatory regime were to break down, could be quickly redirected. One could imagine a version of Plan A that addresses this concern by limiting capabilities primarily via limiting compute instead of algorithms. This has some other benefits as well like being more enforceable than our version of Plan A. 

More generally, I'm excited about scenarios like Plan A that vary anything described in the assumptions supplement.

Finally, I'm also excited about scenarios for plans that are less desirable than Plan A which may be easier to implement. 

Covert Project Further Research 

We present our current best guess modeling of covert projects here. However, this is an area where many on our team had different intuitions during the writing process, and we continue to have substantial uncertainty. Overall, it seems very tractable to significantly improve the analysis with further effort, as well as to identify additional levers to prevent, deter, or detect covert projects.  

Some particularly important areas of further research: 

• Are there near term actions that governments or private actors could take that would dramatically reduce the viability of covert projects? 
• Will it be possible to tell early on whether weight theft can be prevented? In our scenario, we assume that the consortium quickly becomes confident in their respective mitigations and can resume scaling in January 2030. That timeline isn’t loadbearing: if they were obviously not confident in their mitigations, they could remain paused until they were. But it is somewhat loadbearing that consensus can eventually be achieved on whether weight theft is possible.
• Come up with better ways to bound the capabilities gain from distillation. Our current mitigations are here. 
• How much do R&D compute reductions slow takeoff? Our current best guess is that 10x less compute causes the intelligence explosion to go 6x slower (80% CI: 3.5, 8). This modeling is central to effectiveness estimates of any compute governance interventions, which are the primary lever for slowing down AI capabilities progress. 
  • Most of our uncertainty here comes from our uncertainty over the current rate of software progress, so gaining clarity on that would help.

US Domestic Governance

In AI 2040, we briefly discussed some important aspects of US governance. By this I mean both narrow questions about how to structure AI regulation (relevant section here) as well as broader questions about how government and society is structured after extremely advanced AI has been built and diffused into the world. This is a part of AI 2040 we are especially uncertain about because it’s an inherently social and political question, and it’s also further outside our domains of expertise. So, we’d be excited for more people to think about how US domestic governance would evolve over the course of a Plan A scenario and make improved proposals. 

Some example questions to think about:  

• Make a proposal for how to structure an AI regulatory apparatus. Ideally, this regulator would have good epistemics, won’t get captured, has good internal incentives, and is incentive compatible for existing power structures to empower. Our proposal here. 
• Make a proposal for measures to improve generic government functionality. Governments will be under extreme pressure during the intelligence explosion: there will likely be tens of trillions of dollars of economic pressure from industry to avoid regulation, but also immense anti-AI public pressure for regulation. However, it will also be a period of intense social change, and government improvement proposals that are impossible today may become viable. There are a wide range of proposals that I think might be good here ranging from relatively simple (e.g. reforms to make it easier to hire technical talent into the federal government) to extreme changes (e.g. constitutional amendments that massively reform the way the government is structured). 

Verification 

We discuss our current modeling in the Verification Plan, and have a get involved page. 

Some areas I’m excited particularly excited about include: 

• Build inference-only retrofit capacity (see maintext, detailed version). Aspects of this are in progress, but more work needs to be done. 
• Attempt to accelerate “AI Silver Bullet” technologies (See Section 3.1, Method #5 of the covert project supplement), in particular: (i) privacy preserving AI auditing and (ii) lie detectors. We discuss downsides of lie detectors here, but on net, I think that accelerating their development would be good, if it is done in an open / transparent way so that the technology can’t be captured by one actor or government 

Economics 

You can read our supplement here and our Plan A model here. 

There are two very distinct schools of thought with respect to the economics of AI. The mainstream economist view is that the current 3% GDP growth per year trend will continue for the foreseeable future. My view is that we’ll get explosive economic growth driven by superhuman AIs, of the sort that we see in the AI 2040 scenario, except much more extreme if we fail to make appropriate regulations on the intelligence or industrial explosion. 

Unfortunately, in large part because of this divide, very little economics work has been done that takes AGI seriously (though there are a few notable exceptions). Moreover, we have large uncertainty about many aspects of the economy before, during, and after the intelligence explosion. Better modeling would improve our understanding of a crucial aspect of the future, and thus enable us to choose better response strategies. This is especially important in worlds like Plan A; where governance leads to a more drawn out intelligence explosion than I expect, but it's also important in natural slow takeoff worlds.

Some concrete projects on the economics of AI that I’m excited about are: 

• Make a better version of our economics model.  We aren’t economists, and only had time to make a relatively scrappy model. I’m confident it’s possible to do a much better analysis with more time or expertise than we have. 
• Think about robot doubling times in realistic scenarios, e.g. each of plan A-D. The best analysis I’m aware of is this series by Damon Binder. I think a valuable next step would be to try to make a reasonable full fledged model of the industrial explosion, maybe on top of the AI Futures Model. 
• Build a model of what would happen to the AI industry in Plan A vs Plan S vs Plan D, which includes valuations, margins, number of companies at the frontier, etc.

Various Plan A Details 

Here’s a final list of other details which we didn’t have the chance to work out fully during the writing of Plan A, but which would be valuable projects that could strengthen Plan A: 

• In AI 2040 we make a proposal for limiting the persuasive capabilities of AI. However, we aren’t very concrete about the exact operationalization of the policy. To actually implement something like this, we’ll need a concrete set of standards for what is and isn’t allowed. This seems extremely important for preserving human autonomy in an AI age, but also involves some philosophically thorny questions: what exactly is the difference between “helping someone learn new things” and “convincing them that you are right”. 
• Write up the ambitious D/acc plan. During Plan A, there is a massive budget for defense. What’s the best way to spend that budget? Work out the $100T defense plan for the mid 2030s in Plan A for bio/cyber/persuasion and other threats.  
• A key part of the industrial policy in plan A is cap and trade on compute buildout. Unlike GPUs (which can easily be quantitatively compared e.g. in terms of FLOP/s), there’s not a natural notion of performance to cap for robots. Figure out if there’s a good operationalization of robot cap and trade, or figure out an alternate policy for limiting growth that doesn’t rely on robot cap and trade.   
• Work out military arms control in Plan A: should the US and China agree to limit AI superweapons or not? If yes, a defector might be able to gain a military advantage by developing AI superweapons in secret. If no, you might get an AI superweapon arms race which could destroy the planet. We explore this tradeoff briefly in this box. For a more complete answer, it’d be useful to look into the concrete paths towards AI superweapons that might be viable with AIs in the human range, and then examine the offense defense balance, as well as the political feasibility of defense. 
• Look into deal decline base rates more seriously + think about the disanalogies to historical agreements (e.g. Plan A is a much bigger deal). We’d recommend thinking about what we can do to make deal decline less likely.
• We’re much more knowledgeable about AI than the politics of AI. So we’d be interested in folks who actually understand the politics of the situation to do more analysis and thinking about the geopolitics of AGI. In particular, we’d be interested in analyses of: (i) How would the CCP orient to a Plan A-style deal?; and (ii) Are there variants of Plan A that are much easier to accomplish politically, but capture many of the benefits? (e.g. filtered transparency instead of total research transparency).
• Do research into any of the proposals in our AI for epistemics supplement. 
• Investigate and understand how software progress happens. Specifically, if we limit large experiments that require huge amounts of compute, how much does this slow software progress? This informs how well algorithmic progress regulation can work in our scenario, and if it isn’t viable, a different lever may be needed (e.g. limiting overall compute). A related question is how to incentivize scale-dependent algorithmic progress, i.e. algorithms that work well at large scale but not small scale, and thus only help legal projects in Plan A.  

Conclusion 

There are many fewer people than I’d like trying to chart a beneficial path through the intelligence explosion. If you are interested in any of the above questions, I’d encourage you to think about them. 

In fact, we care so much about this that we are considering launching relatively significant monetary prizes for useful follow-up research (which would include progress on any of the ideas above).