And I can’t bet on “what they supposedly imply” because if my beliefs about AGI are right, then I expect we’ll both be much too dead for a bet to pay out in my favor.
I could pay you in advance as I did with Greg Colbourn, although this would only work for relatively short timelines. Otherwise, I had better keep my money invested.
Hi Stephen.
To create the estimates in the tables below I used the following sources (ordered from most to least reliable):
- Web pages listing all the researchers working at an organization.
- Asking people who work at the organization.
- Scraping publications and posts from sites including The Alignment Forum, DeepMind and OpenAI and analyzing the data [2].
- LinkedIn insights to estimate the number of employees in an organization.
I find it interesting that you ranked LinkedIn last. Is this because many people working at the target organisations do not add their roles or organisations to their LinkedIn profiles?
I hope that is a roughly correct rendition of your argument.
Thanks for the great summary, Kave!
So this argument doesn't bite for, say, shrimp welfare interventions, which could be arbitrarily more impactful than global health, or R&D developments.
Nitpick. SWP received 1.82 M 2023-$ (= 1.47*10^6*1.24) during the year ended on 31 March 2024, which is 1.72*10^-8 (= 1.82*10^6/(106*10^12)) of the gross world product (GWP) in 2023, and OP estimated R&D has a benefit-to-cost ratio of 45. So I estimate SWP can only be up to 1.29 M (= 1/(1.72*10^-8)/45) times as cost-effective as R&D due to this increasing SWP’s funding.
... (read more)Here are my even-assuming-outside-view criticisms:
- Even the Davidson model allows that the distribution for interventions that increase the
Thanks for the post! I think genetic engineering for increasing IQ can indeed be super valuable, and is quite neglected in society. However, I would be very surprised if it was among the areas where additional investment generates the most welfare per $:
Thanks for the post, Dan and Elliot. I have not read the comments, but I do not think preferential gaps make sense in principle. If one was exactly indifferent between 2 outcomes, I believe any improvement/worsening of one of them must make one prefer one of the outcomes over the other. At the same time, if one is roughly indifferent between 2 outcomes, a sufficiently small improvement/worsening of one of them will still lead to one being practically indifferent between them. For example, although I think i) 1 $ plus a chance of 10^-100 of 1 $ is clearly better than ii) 1 $, I am practically indifferent between i) and ii), because the value of 10^-100 $ is negligible.
Thanks, JBlack. As I say in the post, "We can agree on another [later] resolution date such that the bet is good for you". Metaculus' changing the resolution criteria does not obviously benefit one side or the other. In any case, I am open to updating the terms of the bet such that, if the resolution criteria do change, the bet is cancelled unless both sides agree on maintaining it given the new criteria.
Thanks, Dagon. Below is how superintelligent AI is defined in the question from Metaculus related to my bet proposal. I think it very much points towards full automation.
... (read more)"Superintelligent Artificial Intelligence" (SAI) is defined for the purposes of this question as an AI which can perform any task humans can perform in 2021, as well or superior to the best humans in their domain. The SAI may be able to perform these tasks themselves, or be capable of designing sub-agents with these capabilities (for instance the SAI may design robots capable of beating professional football players which are not successful brain surgeons, and design top brain surgeons which are not football players). Tasks
This is a linkpost for How should we analyse survey forecasts of AI timelines? by Tom Adamczewski, which was published on 16 December 2024[1]. Below are some quotes from Tom's post, and a bet I would be happy to make with people whose AI timelines are much shorter than those of the median AI expert.
The Expert Survey on Progress in AI (ESPAI) is a large survey of AI researchers about the future of AI, conducted in 2016, 2022, and 2023. One main focus of the survey is the timing of progress in AI1.
[...]
This plot represents a summary of my best guesses... (read 825 more words →)
Fair! I have now added a 3rd bullet, and clarified the sentence before the bullets:
... (read more)I think the bet would not change the impact of your donations, which is what matters if you also plan to donate the profits, if:
- Your median date of superintelligent AI as defined by Metaculus was the end of 2028. If you believe the median date is later, the bet will be worse for you.
- The probability of me paying you if you win was the same as the probability of you paying me if I win. The former will be lower than the latter if you believe the transfer is less likely given superintelligent AI, in which case the
Thanks for the great questions!
Here is the talk. I liked it!