In this paper, we make recommendations for how middle powers may band together through a binding international agreement and achieve the goal of preventing the development of ASI, without assuming initial cooperation by superpowers. You can read the paper here: asi-prevention.com In our previous work Modelling the Geopolitics of AI,...
We model how rapid AI development may reshape geopolitics in the absence of international coordination on preventing dangerous AI development. We focus on predicting which strategies would be pursued by superpowers and middle powers and which outcomes would result from them. You can read our paper here: ai-scenarios.com Attempts to...
Expert opinions about future AI development span a wide range, from predictions that we will reach ASI soon and then humanity goes extinct, to predictions that AI progress will plateau soon, resulting in weaker AI that presents much more mundane risks and benefits. However, non-experts often encounter only a single...
control granularity might be higher, and permanence might be higher
That's pretty much it, just to an extreme extent. Given that ASI could be extremely powerful and that it's really hard to predict what it could do, I recommend thinking of it as:
I keep trying to map this onto Canada's situation and getting stuck. We're mid-negotiation with the US on trade and political capital is finite. How does leadership spend it on ASI risks most Canadians aren't thinking about?
IMO the key here is "most Canadians aren't thinking about", this can be changed through awareness campaigns. Most people aren't aware that AI companies are shooting for ASI, and wouldn't like it if they knew.
First, timing ... If US or Chinese leadership believes ASI is 2-5 years out, they'll absorb enormous economic costs for a shot at decisive advantage
I think this is reasonable, which is why we include the more extreme measures including recognition of the right to self-defense. I personally would be surprised if we could throw this together so quickly that none of the conditional deterrence measures ever need to be activated...
In darker timelines, I think the more extreme economic measures could slow down the superpower AI programs and give time for middle powers to get more serious with their military deterrence, which has a good chance of being effective imo.
... (read more)The US has extensive
In this paper, we make recommendations for how middle powers may band together through a binding international agreement and achieve the goal of preventing the development of ASI, without assuming initial cooperation by superpowers.
You can read the paper here: asi-prevention.com
In our previous work Modelling the Geopolitics of AI, we pointed out that middle powers face a precarious predicament in a race to ASI. Lacking the means to seriously compete in the race or unilaterally influence superpowers to halt development, they may need to resort to a strategy we dub “Vassal’s Wager”: allying themselves with a superpower and hoping that their sovereignty is respected after the superpower attains a DSA.
Of course, this requires... (read 647 more words →)
We model how rapid AI development may reshape geopolitics in the absence of international coordination on preventing dangerous AI development. We focus on predicting which strategies would be pursued by superpowers and middle powers and which outcomes would result from them.
You can read our paper here: ai-scenarios.com
Attempts to predict scenarios with fast AI progress should be more tractable compared to most attempts to make forecasts. This is because a single factor (namely, access to AI capabilities) overwhelmingly determines geopolitical outcomes.
This becomes even more the case once AI has mostly automated the key bottlenecks of AI R&D. If the best AI also produces the fastest improvements in AI, the advantage of the leader... (read 500 more words →)
Believe what?
That's kind of the point... people reject premises all the time. And I don't mean in the "this premise seems unrealistic" sense. I mean more in the "I refuse to participate in this thought experiment!" sense. Even when the point wasn't to give a lesson about the shape of reality but to give a lesson about the shape of the reader's mind and how they respond to the thought experiment.
People just hate inspecting their own mind with a passion. It's also common for people not to trust you when you suggest a thought experiment if they can't see where it's going. It's very easy to get an anger reaction this way (literally raised voice, tense muscles, raised heartbeat type of reaction).
Expert opinions about future AI development span a wide range, from predictions that we will reach ASI soon and then humanity goes extinct, to predictions that AI progress will plateau soon, resulting in weaker AI that presents much more mundane risks and benefits. However, non-experts often encounter only a single narrative, depending on things like their social circles and social media feed.
In our paper, we propose a taxonomy that divides expert views on AIs into three main clusters. We hope to provide a concise and digestible way to explain the most common views and the crucial disagreements between them.
You can read the paper at: doctrines.ai. Here is a brief summary:
Dominance doctrine. The... (read 281 more words →)
For concreteness, let’s say that the world model requires a trillion (“N”) bits to specify, the intended head costs
10,000 bits, and the instrumental head costs 1,000 bits. If we just applied a simplicity prior directly, we expect to spend N + 1,000 bits to learn the instrumental model rather than N + 10,000 bits to learn the intended model. That’s what we want to avoid.
Not sure if I'm misunderstanding this, but it seems to me that if it takes 10,000 bits to specify the intended head and 1000 bits to specify the instrumental head, that's because the world model - which we're assuming is accurate - considers humans that answer a question with... (read more)
This sounds to me like a very weak excuse. If you change your mind on something this important (for example, you are now confident alignment is easy despite expecting RSI in a couple of years and doing your best to accelerate it), you had better say... (read more)