We made a map of the doom debate

Sean Herrington; Paul Hindoian; mikaelacankosyan; David Bravo; keivnc; Josh Tuffy; Christopher Davis; Khai Tran; Maryam Hampaei

This was produced as a part of the AI Safety Camp 2026 "Assumptions of the Doom Debate" project, led by Sean Herrington, who was also the lead author on this post. The other participants have equal contributions and are listed in no particular order. It is the first in a sequence we intend to publish over the coming weeks.

TL;DR:

We have created a breakdown of AI threat pathways, which can be accessed at https://lifeuniversesafety.com/doom-assumptions/index.html
This breakdown is in a tree format, and we allow people to set their own probabilities for each threat pathway
You can use this to drill down into components of your P(Doom)
You can analyse the sensitivity of your beliefs to changes in your assumptions
You can compare your worldview to others’ and find cruxes automatically
The exercise of creating this sort of structure is valuable as a way to think about the future in a more general manner.

Introduction

Just about everyone in the AI community seems to disagree about the risks of the technology. People disagree on the likelihood (Yann LeCun: <0.01% chance of extinction; Roman Yampolskiy: >99.99%), the worst threat (AI takeover vs concentration of power vs gradual disempowerment vs bioweapons vs …), the timelines and many other things. It seems like a shared framework would be helpful in this capacity.

The main issue that comes with creating such a thing is that it requires everyone to agree on it. We call this requirement "worldview independence"; it significantly narrows down the space of options for the shape this framework can take. In particular, every part of the structure has to be a mathematical statement; those are some of the only things people can agree on.

The structure we eventually ended on looks like this:

Breaking it down:

E: Event that we're worried about (Nuclear war, Bioweapon is released, Human extinction, etc)
A: Set of worlds, e.g., "AI-driven worlds" = "credence that we live in a world where AI is making E more likely."
¬A: Set of worlds which are not in A, e.g., "Non-AI worlds" = "credence that we live in a world where AI is not making E more likely"
P(E|A): Probability that event E happens in worlds A.

On the website, we also have a timeframe, such that we are actually talking about event E happening "within 30 years", for example. Putting real values in looks like this:

In this case, the thing we want to know is our probability of an existential catastrophe occurring within the next 30 years. We split this into AI-caused and not AI-caused.^[1] You can then set your probabilities for each of these and calculate your overall final probability that an existential catastrophe happens.

This becomes significantly more interesting once it gets scaled, as it enables you to break the future down into individually visualisable threat paths. We’ve also found as a group that the exercise of creating this tree has helped us see a range of possibilities we had not considered.

For instance, we hadn’t previously thought much about scenarios where powerful and otherwise aligned AIs made mistakes, but after further thought, consider it significantly more likely.^[2] This insight was directly generated by the thought that while trying to split worlds apart, we were implicitly assuming that dangerous AI scenarios were caused by AIs which wanted these scenarios to come about.

User Guide

Labels

Labels are simply changes that can be made to the text in the tree, allowing you to change what you’re talking about. They appear in the left sidebar, next to the tree:

We have 4 different labels you can apply to your tree.

Label	Description
Danger	Changing the danger label is equivalent to changing what you are talking about: some people want to ask whether we’re going to have an “existential catastrophe"; for others, the question is whether “everyone dies" or whether “a bioweapon gets deployed”.
Timeframe	One of the biggest debates in the AI community is one of timelines. Our tree is therefore based around the question of whether the danger occurs within the timeframe of "x years". Use this label to change which timeframe you are discussing.
Author	Put your name here to show people that it’s your worldview rather than anyone else’s.
Perspective	You can use this to change whether you are talking about your inside view (where you are basing your opinion on a gears-level internal model) or your outside view (where you also include meta-level factors outside that model).

Base Tree

Our base tree has 4 different branching points, which we’ll explain briefly here.

Branch	Description
AI driven/Not	Separates out worlds in which AI worsens the danger from those in which it doesn’t.
Single AI/Multipolar (AI driven world)	Separates out worlds where most of the danger comes from a single AI from worlds where it comes from multiple.
Internal Model/Not (Single AI Danger world)	Separates out worlds where the single AI causing the most danger is an AI with an internal model of the danger it’s causing from one where it doesn’t. An example of a dangerous system without a model could include (for instance) a simple missile detection system entrusted by a nuclear power with retaliation. The main question here is, "How likely is it that the AI knows what it's doing?"
Expects danger/doesn't (Internal Model world)	Splits worlds where the single most dangerous AI expects the danger it is causing to occur (such as in traditional misaligned superintelligence narratives) from those in which it does not (for instance, because the AI has itself made a mistake).

Tree features

Pin branch overrides

If you do not want to set all of the nodes in the tree, you can override the leaves by simply moving the probabilities higher up the tree. You will still be able to edit the probabilities further down, but they will no longer affect your root probability.

Clicking on the triangle symbol allows you to switch between the set probability for that node and the propagated value for the nodes further down the tree.

Collapse tree

The +/- icons in the corner of the nodes of the tree can be used to expand and collapse the tree below them.

Sub-branch probabilities

Each node below the top section of the tree has two numbers: The main one is the actual probability of this world existing and causing an AI catastrophe.

Screenshot 2026-05-21 115113 w circle 2.png

The number below is the same, but conditioned on us already being in that branch of the tree.

Screenshot 2026-05-21 115113_w circle.png

Connector thickness

The lines between nodes get thicker for branches carrying more probability mass through worldspace. This is a representation of how the worlds are split, rather than representing probabilities of danger.

Sensitivity Analysis

The sensitivity analysis is the derivative of your overall probability by each individual value you’ve set. It allows you to see where changing your mind would be the most significant.

Crux Analysis

Different people have different worldviews. Crux analysis is a way to see which facts are the most important sources of disagreement.

We are comparing 2 worldviews, A and B. These can either be preset or user-saved.

For each node, there are 2 bars:

The green bar is how much A's final probability would move towards B's if B convinced them of their worldview in that particular node
The amber bar is how much B's would change were A to convince them.

Bars going to the right close the gap in root probabilities, while those going to the left open them.^[3]

For instance, in the picture above, B convincing A that multipolar AI means a 15% probability of existential catastrophe rather than 10% would move A's overall probability by 2.2 percentage points towards B's.

A convincing B to lower their probability to 10% rather than 15% would move B's overall probability by 2.4 percentage points towards A's.

Ranges

We have added ranges to the site. This allows for a representation of your uncertainty in the final result: if your current probability estimate is 60% for something, but it seems plausible that on another day you might put 50 or 70, it can be useful to know how that would affect the final result.

We currently have 2 range propagation modes:

Worst case: treats the ranges as maximum and minimum possible probabilities
Independent: treats the ranges as 10th and 90th percentiles and performs a Monte Carlo simulation^[4] to establish subsequent ranges.

We would generally recommend using independent propagation as 'better', but the worst case can also be useful as a view of the full range of possibilities you see as plausible.

Uncertainty Reduction

When using ranges, you get the additional option of using uncertainty reduction.

This shows you how much reducing your uncertainty on individual nodes reduces your overall uncertainty.

Other features

Feature	Description
Worldviews	The tree contains some preset worldviews, which are AI-generated estimates of prominent figures' worldviews, inspired by their public statements. Please note these are not endorsed by the actual figures and are there for interest purposes only.
Share	You can share worldviews with friends. Click on the button to get a shareable link they can then paste in their browser.
Save	Save this worldview as a JSON file and to local memory. Saved worldviews persist across sessions.
Import	Import a JSON worldview file.
Reset	Reset all values to 50%
Delete	Delete a saved worldview

^{^}
It's hard to define this precisely, and we've had a bunch of debates about this ourselves. The basic intuition is, for the example E=Existential risk, "How likely do you think it is that we live in a timeline where AI will overall contribute to existential risk (for instance, because it is misaligned) vs helping to mitigate it (via, e.g., helping to solve problems associated with climate change)?"
^{^}
For a brief intuition here, consider this statement from the perspective of a chimpanzee: “Humans are smart enough to go to the moon, change animals’ DNA and communicate instantly across the globe. There’s no way they’d be dumb enough to cause climate change; they live on this planet like we do!”
^{^}
An example of how agreeing on one node can make your final probabilities diverge: A is broadly more pessimistic than B, except on one node. Agreeing with B on that node would make A even more pessimistic, taking their overall probability away from B's.
^{^}
The individual probabilities are drawn from a beta distribution fit to the percentile range

^{^}

Which is itself determined by other factors, which you might quantify if you're doing a more in-depth version of this exercise.

^{^}

Which is itself determined by other factors, which you might quantify if you're doing a more in-depth version of this exercise.

[-]MichaelDickens4h54

This is a nice exercise! Pretty clean and understandable UX too.

On the object level, I think some of the nodes don't capture distinctions that I consider important. Like the question of whether ASI has a world model of existential catastrophe seems basically irrelevant to me. It looks to me like you added that node as a response to things Yann LeCun has said about why his P(doom) is low?

It's also missing some question that seem pretty important, like, will takeoff be slow or fast? How likely are extrapolations of current alignment techniques to work? How likely are governments to start taking AI extinction risk seriously? (Of course you can't add too many questions or it gets unwieldy.)

[-]StanislavKrym3h20

What would you or the OP's authors say about the AI futures map by swante? Does anyone remember other such flowcharts?

[-]Andrii Vasylenko1h10

The thing that stood out to me was the sheer number of subquestions. I think including too many is counterproductive because of the conjunction fallacy.

[-]Andrii Vasylenko2h32

It's possible that I'm missing something, but the last two nodes (the AI knows that x-risk is possible, and what the AI's beliefs are about) don't seem like they are at all relevant to x-risk forecasting.

For me specifically, the beliefs that are the most entangled with x-risk probability are whether a sufficiently strong ban treaty (e.g.) will be put into place^[1], since it gives time for more safety research and potentially human intelligence amplification, and the difficulty of the alignment problem (although this is kind of hard to estimate).

Other factors like the safety-mindedness of the lab(s) that get to AGI first, whether the leadership structure is basically altruistic, whether takeoff will be fast or slow, etc. are less important on my model, although I still think they should be considered.

^{^}
Which is itself determined by other factors, which you might quantify if you're doing a more in-depth version of this exercise.

[-]ajskateboarder36m10

Unrelated: the "AI knows x-risk is possible" node seems intuitively related to strategic competence (though in this map, higher probability increases final p(doom))

[-]Andrii Vasylenko7m10

There's a difference between an LLM saying that "x-risk is possible" when you ask it the question, and the sort of understanding that propagates to actions like refusing to work on capabilities research. (see also)

19