Aren’t most AI safety researchers focused on product safety? It’s possible that a lot of research is dual-use, or only improves deployability, or is just not relevant to x-risk. Or do you account for this in your extinction reduction numbers already?
I also wonder how things like governance or advocacy fit in here? I wonder what a BOTEC for what progress on the margin towards eg an international pause treaty looks like. Perhaps I should do that.
Yeah, I mean all the numbers here have gigantic error bars right. The point is broadly to put a ballpark number on the value of the work you're doing. In terms of dual-use, I would expect some people's work to be net negative, but everyone seems to disagree on whose work exactly. There's O(2000) people working on AI Safety right now (source: multiple separate conversations have given the same or similar numbers), so that is what I was basing it off (plus adjustment for the field growing). If you think e.g only 200 of those people are doing anything worthwhile, feel free to adjust the estimate to account for it.
I don't know what the numbers are for governance and/or advocacy, but I think it would be great if you could have a look!
When thinking about AI risk, I often wonder how materially impactful each hour of my time is, and I think that this may be useful for other people to know as well, so I spent a couple of hours making a couple of estimates. I basically expect that a tonne of people have put a bunch more time into this than me, but this is nice to have as a rough sketch to point people to.
I'm going to make 3 estimates: an underestimate, my best-guess estimate and (what I think is) an overestimate.
Starting facts[1]:
I am going to commit statistical murder and assume this means that everyone on the planet lives ~42.7 years from this point onwards.
Since the population is growing, we should take that into account:
Given these parameters, we can figure out the total expected years of life we care about for each scenario:
Median:
Current population: 60 years x 8.3 B
Additional population (linear approximation):
Additional population life span: 73.8 years + ~1/3yrs added/year = 110 years
Total expected years of life:
I think it might be best to skip out on the overestimate. For the underestimate, we'll go with ~20 years of research to produce a 1% chance of a 1% decrease in the final risk for the entire field. Extinction occurs 30 years from now. For the median estimate, we'll go with 5 years of research to reduce a risk of extinction, which happens 10 years from now, and we will go with a 50% chance of a 5% reduction in risk.
Expected years of life available to be saved:
Expected years of life actually saved:
Number of AI Safety researchers:
Expected impact per researcher:
We've said the researchers have 20/5 years to make an impact, which gives us:
Going back to the ~40 years of life expected for the modern median human, this gives an underestimate of 1 year of work to save one life, or a median estimate of 5 mins/life. This is a pretty broad range funnily enough.
1 year of work to save one life is just a tad worse than the 1.2 lives/year saved donating £3000/year as advertised by Effective Altruism UK. If we take that value as given and assume 1 life = £2500, this means that on the median estimate, you should be earning £2500 x 10^6 / 40 = £62.5 million/ year. If only the world was more sensible.
All population data comes from https://www.worldometers.info