A summary and review of Bostrom's Superintelligence is in the December issue of Reason magazine, and is now posted online at Reason.com.

A summary and review of Bostrom's Superintelligence is in the December issue of Reason magazine, and is now posted online at Reason.com.

I remain impressed by how much awareness one high-status academic can raise by writing a book.

My first thought upon reading this was "Holy crap! Really?!" as I began revising my own probability/risk estimates. Then I realized that people probably also said this in the 1950s. How much are other people updating on this?

This looks to me like a misunderstanding of Müller & Bostrom 2014. The actual figure is that 50% of AI researchers give a 10% probability of HLMI by 2022.

Müller, V. C., & Bostrom, N. Future progress in artificial intelligence: A survey of expert opinion. In V. C. Müller (Ed.), Fundamental Issues of Artificial Intelligence. Berlin: Springer. 2014

That's what I thought at first, too, but then I looked at the paper, and their figure looks right to me. Could you check my reasoning here?

On p.11 of Vincent's and Nick's survey, there's a graph "Proportion of experts with 10%/50%/90% confidence of HLMI by that date". At around the the 1 in 10 mark of proportion of experts -- the horizontal line from 0.1 -- the graph shows that 1 in 10 experts thought there was a 50% chance of HLAI by 2020 or so (the square-boxes-line), and 1 in 10 thought there was a 90% chance of HLAI by 2030 or so (the triangles-line). So, maybe 1 in 10 researchers think there's a 70% chance of HLAI by 2025 or so, which is roughly in line with the journalist's remark.

Did I do that right? Do you think the graph is maybe incorrect? I haven't checked the number against other parts of the paper.

There's a good chance that the reviewer got the right number by accident, I think, but it doesn't seem far enough away to call out.

Gosh, you might be right! I'm surprised by that. Would be good to have the data to get into it in more detail.