About me and this review: I don’t identify as a member of the rationalist community, and I haven’t thought much about AI risk. I read AstralCodexTen and used to read Zvi Mowshowitz before he switched his blog to covering AI. Thus, I’ve long had a peripheral familiarity with LessWrong. I picked up IABIED in response to Scott Alexander’s review, and ended up looking here to see what reactions were like. After encountering a number of posts wondering how outsiders were responding to the book, I thought it might be valuable for me to write mine down. This is a “semi-outsider “review in that I don’t identify as a member of this community, but I’m not a true outsider in that I was familiar enough with it to post here. My own background is in academic social science and national security, for whatever that’s worth. My review presumes you’re already familiar with the book and are interested in someone else’s take on it, rather than doing detailed summary.

My loose priors going in:

AI poses an obvious and important set of risks (economically, socially, politically, military, psychologically etc.) that will need to be carefully addressed. Some of those have already arrived, and we need to think seriously about how we will manage them. That will be difficult, expensive, and likely to greatly influence the profitability of AI companies.
“Existential” risk from AI (calling to my mind primarily the “paperclip maximizer” idea) seems relatively exotic and far-fetched. It’s reasonable for some small number of experts to think about it in the same way that we think about asteroid strikes. Describing this as the main risk from AI is overreaching.
The existential risk argument is suspiciously aligned with the commercial incentives of AI executives. It simultaneously serves to hype up capabilities and coolness while also directing attention away from the real problems that are already emerging. It’s suspicious that the apparent solution to this problem is to do more AI research as opposed to doing anything that would actually hurt AI companies financially.
Tech companies represent the most relentless form of profit-maximizing capitalism ever to exist. Killing all of humanity is not profitable, and so tech companies are not likely to do it.

To skip ahead to my posteriors:

Now, I’m really just not sure o

...

Posts

Wikitag Contributions

Comments