Crossposted to the EA Forum.
In October 2022, 91 EA Forum/LessWrong users answered the AI timelines deference survey. This post summarises the results.
The survey was advertised in this forum post, and anyone could respond. Respondents were asked to whom they defer most, second-most and third-most, on AI timelines. You can see the survey here.
This spreadsheet has the raw anonymised survey results. Here are some plots which try to summarise them.
Simply tallying up the number of times that each person is deferred to:
The plot only features people who were deferred to by at least two respondents.
Some basic observations:
One thing we tried was computing a “weighted” score for each person, by giving them:
If you redo the plot with that score, you get this plot. The ordering changes a bit, but I don’t think it really changes the high-level picture. In particular, the basic observations in the previous section still hold.
We think the weighted score (described in this section) and unweighted score (described in the previous section) are the two most natural metrics, so we didn’t try out any others.
Yeah, we do think some people have highly correlated views, in the sense that their views depend on similar assumptions or arguments. We tried plotting the results using the following basic clusters:
Here’s what you get if you simply tally up the number of times each cluster is deferred to:
This plot gives a breakdown of two of the clusters (there’s no additional information that isn’t contained in the above two plots, it just gives a different view).
This is just one way of clustering the responses, which seemed reasonable to us. There are other clusters you could make.
Sam and McCaffary decided what summary plots to make. McCaffary did the data cleaning, and wrote the code to compute summary statistics/make plots. Sam wrote the post.
Daniel Kokotajlo suggested running the survey. Thanks to Noemi Dreksler, Rose Hadshar and Guive Assadi for feedback on the post.
You can see the code for these plots here, along with a bunch of other plots which didn’t make the post.
Here’s a list of people who were deferred to by exactly one respondent.
Arguably, Metaculus doesn’t quite fit with “everyone else”, but I think it's good enough as a first approximation, especially when you also consider the plots which result from the weighted score (see next section).
This cluster could have many other names. I’m not trying to make any substantive claim by calling it the Open Philanthropy cluster.
Did people say why they deferred to these people?
I think another interesting question to correlate this would be "If you believe AI x-risk is a severely important issue, what year did you come to believe that?".
No, only asked respondents to give names
Agree, that would have been interesting to ask
Things that surprised me about the results
I don't remember if I put down "inside view" on the form when filling it out but that does sound like the type of thing I may have done. I think I might have been overly eager at the time to say I had an "inside view" when what I really had was: confusion and disagreements with others' methods for forecasting, weighing others' forecasts in a mostly non-principled way, intuitions about AI progress that were maybe overly strong and as much or more based on hanging around a group of people and picking up their beliefs instead of evaluating evidence for myself. It feels really hard to not let the general vibe around me affect the process of thinking through things independently. Based on the results, I would think more people thinking about this for themselves and writing up their reasoning or even rough intuitions would be good. I suspect my beliefs are more influenced by the people that ranked high in survey answers than I'd want them to be because it turns out people around me are deferring to the same few people. Even when I think I have my own view on something, it is very largely affected by the fact that Ajeya said 2040/2050 and Daniel Kokotajlo said 5/7 years, and the vibes have trickled down to me even though I would weigh their forecasts/methodology less if I were coming across it for the first time. (The timelines question doesn't feel that important to me for its own sake at the moment but I think it is a useful one to practise figuring out where my beliefs actually come from)