This is an attempt at building a rough guess of the impact you might expect having with your career in AI safety.

Your impact will depend on the choices you make, and the estimation of their consequences also depends on your beliefs about AI safety, so I built an online tool so that you can input your own parameters into the model.

The main steps of the estimation

Compute when a world-ending AGI might happen, if there has been no AI safety progress,
Compute when the AI alignment problem might be solved,
Compute your impacts on timelines, and see how it affects the odds that humanity survives.

Main simplifying assumptions

Unsafe AGI being built and AI alignment being solved are independent events: progress towards AGI doesn’t speed up AI safety, but AI Alignment prevents building an unsafe AGI.
AI alignment is either solved or not: if it is solved before an AGI can be built, nothing happens, if it is not solved when an AGI is built, and AGI is unsafe by default, then humanity disappears. Under this assumption, AI alignment is "solved" if a technical solution is found and the alignment tax is low enough that everyone pays it.
Anything that happens after 2100 is irrelevant (AI alignment can’t be solved after this point, and AGI can’t be built beyond this point).
There is no uncertainty on how much you speed up progress on AI alignment. If your impact is small, using your expected speedup on AI alignment as the “true value” of your speedup should not make a big difference in the end result. However, it would make the estimate completely wrong in more extreme scenarios.

These assumptions are very strong, and wrong. I’m unsure in which direction the results would move if the assumptions were more realistic. I am open to suggestions to replace parts of the model in order to replace these assumptions with assumptions that are less wrong. Please tell me if I introduced a strong assumption not mentioned above.

Mathematical model of the problem

Model of when a world-ending AGI might happen, if there has been no AI safety progress

Without your intervention, and if there is no AI safety progress, AGI will happen at time $X$ , where $X$ follows a distribution of density $^q$ over $[t_{0}, + \infty [$ , where $\int_{t_{0}}^{+ \infty}^q = P (AGI happens)$ ^[1]. AGI kills humanity with probability $p_{kill}$ . This is assumed to be independent of when AGI happens. Therefore, without you, AGI wi...

Fermi Estimation