(This is a cross-post of my blog post at Crunch Time for Humanity: https://haggstrom.substack.com/p/a-friction-in-my-dealings-with-friends) 

A few months ago I was invited to a panel discussion whose title (translated from Swedish) was AI: opportunities and fears. I didn’t quite like the ring of this, because it seemed to me that “fears” could be read as a suggestion that the kind of AI risk I like to talk about at public events is mostly just in my head. My reply to the organizers was therefore something along the lines of “I would be happy to participate, but only if you change the title to AI: opportunities and risks, because I want to focus on the actual risks, the facts and the evidence we are facing, not on fluffy talk about fears and other emotions” — a change they were fine with.

Given the aversion to touchy-feely AI discussions that I expressed then, the fluffy, emotion-laden musings of the present blog post will perhaps come as a surprise. But here we go.

If the psychology I am about to describe rings familiar to some readers, I would be very interested to hear about it in the comments. But preferably only from those of you who are on board with the idea of existential AI risk as a real thing. This is not just because these readers are the most likely to have first-hand experience with the kind of psychology I have in mind, but also because comments (no matter how kindly and empathically phrased) from those who are not yet on board are likely to contribute to the exactly the sort of annoying friction I will come to in a moment.

Anyway, enough throat-clearing. The social situation I have in mind, and which happens to me relatively frequently, is when I have a friendly chat with a friend or acquaintance who has not bought into my view of the urgent reality of risks arising from the possibility of a superintelligent AI deciding that it wants to wrest control over the world from humanity; this person knows, however, how engaged I am (professionally and otherwise) with this topic. Since a standard turn in friendly chats is “what have you been up too lately?”, it is perfectly normal and very much to be expected that they ask me about my work on AI risk. Such a conversational direction almost inevitably leads to me painting a somewhat dark image of the situation facing humanity and the prospects of finding a good solution, because I refuse to whitewash about this, and I certainly don’t want to give any false impression that I have had (or am about to have) any significant success in my ambition to help mitigate the risk, or even in the subgoal of raising public awareness of the risk. Usually, the darker the discussion gets, the more kindness and compassion does my conversation partner feed into the conversation.

And this is often the point at which I become annoyed. Which seems kind of bad of me — because being met with kindness and compassion is not the sort of thing that one ought to be annoyed with. But my annoyance comes from having a sense that the problem that my conversation partner is addressing is not AI risk itself (which they don’t think is real) but my state of mind.^[1] [2] Typically they don’t need to be as blunt as saying “it must be tough living with this fear that the world is about to end”, because this can be expressed with subtler cues.

What happens next varies, not just because the details of the conversation and my exact relation to the conversation partner vary, but also because I have deliberately tried different strategies. I have yet to find an approach that I am happy with. The smoothest way is to steer away from the topic under discussion and move to some lighter conversation topic, but this is unsatisfactory because AI risk and how we view it and address it is a tremendously important topic that I ought to take every opportunity to discuss, rather than avoiding it out of convenience.

Another option is to go into further detail about what I and others can do to mitigate existential AI risk, while ignoring all invitations to discuss my personal psychology. However, from the point of view of my conversation partner there is no real AI risk problem to be solved, and a typical consequence of this is that their side of the conversation will not be very constructive. So it sometimes happens that when the discussion enters the realm of AI governance (as it nowadays tends to do fairly quickly, because as opposed to just 4-5 years ago I no longer believe that technical AI alignment on its own has much chance of saving us from AI catastrophe without assistance from politics and legislation), they will bombard me with questions such as “What about China?”, “What about Trump?”, “What about the relentless market forces?”, and so on. These are all valid questions, but as the deck is currently stacked they are also extremely difficult,^[3] to the extent that even a moderately clever discussion partner who is not interested in actually solving the problem but merely in playing devil’s advocate is likely to win the argument and conclude that I have no clear and feasible plan for avoiding catastrophe, so why am I wasting people’s time by going on and on abut AI existential risk?^[4]

A third option is to declare explicitly the way that I think the conversation partner and I are talking past each other, namely that while I’m talking about the global risk caused by AI, they are talking about the very local problem of what this (perceived) risk does to my mind. I will then go on to explain my insistence on steering the conversation towards the former and much bigger problem by pointing out that talking about the latter problem is to focus on a comparatively trivial symptom rather than on the underlying cause. It’s not that I am unaware of the possibility that my personal well-being might improve if I think less about short AI timelines and AI risk, but I am offended by suggestions that this is a solution: worsening my epistemics via (say) religion or opium would contribute nothing to solving AI risk, and it would be antithetical to who I am.

This can easily take the discussion in a similar direction as in option two above, with a possible difference being that I will repeatedly interject something along the lines of “you keep talking about this as a problem that it falls upon me to solve, while in reality we are all sitting in the same boat with respect to existential AI risk, so that you in fact have as much reason as me to try to work towards a solution where we are not all murdered by superintelligent AIs a few years down the road”. However, on at least one occasion where I’ve employed this option, the conversation turned sour.

I am honestly unsure about how to handle these conversations, given the twin goals of keeping them pleasant and of not missing out on any opportunity to convince my conversation partner about the reality of AI risk and the need to do something about it.

1. ^{^}

   Note the similarity with my reaction to the panel discussion title I started out with complaining about.

2. ^{^}

   In a recent LessWrong post, Eliezer Yudkowsky describes a sitaution not entirely unlike mine:

   “How are you coping with the end of the world?” journalists sometimes ask me, and I sometimes reply, “By remembering that it’s not about me.” They have no hope of understanding what I mean by this, I predict, because to them I am the subject of the story and it has not occurred to them that there’s a whole planet out there too to be the story-subject. I think there’s probably a sense in which the Earth itself is not a real thing to most modern journalists.

   The journalist is imagining a story that is about me, and about whether or not I am going insane, not just because it is an easy cliche to write, but because personality is the only real thing to the journalist.

3. ^{^}

   One of my current favorite texts about this extremely difficult situation and how to think in a structured and constructive way about it is Early US policy priorities for AGI by Nick Marsh over at AI Futures Project.

4. ^{^}

   Most of this paragraph is taken from my earlier text Pro tip on discussions about AI xrisk: don’t get sidetracked, which continues:

   And here’s the thing. Many of those who play the devil’s advocate in this way will be aiming for exactly that turn of the conversation, and will implicitly and possibly unconsciously believe that at that point, they have arrived at a reduction ad absurdum where the assumption that AI xrisk is real has been shown to be absurd and therefore false. But the reasoning leading to this reductio doesn’t work, because it relies on (something like) the assumption that the universe is a sufficiently benign place to not put humanity in a situation where we are utterly doomed. Although this assumption is central to various Christian thinkers, it is in fact unwarranted, a horrible realization which is core to the so-called Deep Atheism of Eliezer Yudkowsky, further elaborated in recent work by Joe Carlsmith.

   To reiterate, I do think that the AI governance questions on how to stop actors from building an apocalyptically dangerous AI are important, and I am very interested in discussing them. They are also difficult — difficult enough that I don’t know of any path forward that will clearly work, yet we have far from exhausted all such possibilities, so the challenges cannot at this stage be dismissed as impossible. I want to explore potential ways forward in intellectual exchanges, but am only prepared to do it with someone who actually wants to help, because the field is so full of real difficulties of which we who work in it are so highly aware that our primary need is not for additional devil’s advocates to repeat these difficulties to us. Our primary need is for the discussions to be serious and constructive, and for that we need discussion partners who take seriously the possibility of AI xrisk being real.