An odd aspect of discussing serious threats is the amount of concern people express about you causing other people to be concerned. This kind of makes sense for interlocutors who don’t believe in the threat itself, or think it is overblown (though in that case it is perhaps strange to focus on altruistic concern for potential frightened onlookers rather than the object-level disagreement). But often the person is not actually disputing the threat, they purportedly just want to protect the public from fear, or avoid causing ‘panic’.
A memorable case: on what was to be one of the last normal weekends in 2020, I took an Uber with a friend to an event. On the way we discussed the rising warnings of an international pandemic and our preparations. But my friend wanted us to talk more discreetly, lest we scare the Uber driver.
Why on Earth would it be bad to scare the Uber driver? My friend believed as much as I did that a real and deadly virus was spreading and there was an imminent risk of this affecting us all, including the Uber driver. Didn’t the Uber driver have an interest in knowing about it? Wasn’t it, if anything, our responsibility to tell the Uber driver? Is the concern that, as a normal person, the Uber driver is incompetent to manage himself, and will just scream and run around or buy poorly selected prepper equipment?
In conversations about AI risk, I sometimes see the same thing. Geoffrey Hinton says he thinks there’s a 10-20% chance of human extinction, and some people seem genuinely most concerned is that maybe the press didn’t add enough disclaimers about the process by which he reached that number, and the public may get unnecessarily worried. I agree it would be non-ideal if people were 20%-level worried when they would only endorse being 7% worried on further methodological inspection. But among non-ideal aspects of a situation where most of the relevant scientists believe their field is heading toward a modest-to-strong shot at killing us, it’s interesting to rate “maybe people will be too concerned” as a top concern.
What is going on? In this particular case, I could imagine behaving this way if the original communication seemed dishonest. But finding this dishonest seems similar to complaining if someone yells “fire!” that they should have yelled “I subjectively guess that it’s highly likely there’s a fire because of the smoke and flames but I’m not an expert”. And it seems like there’s something else going on with wanting that.
A related phenomenon: people casually mention ‘causing a panic’ as a thing that is assumed to be too terrible. Like, yes there’s some upside to warning people that there is a major threat to their lives that they can do something about. Maybe doing that will help stop the world from ending. But! What if they get all emotional? They may not act in the most clear-eyed and rational way. They may talk to each other in epistemically unvirtuous fashions and get even more concerned. They may buy too much toilet paper or run on a perfectly functional bank or protest for poorly designed policies.
I mean, indeed this is all worse than them addressing threats in the most rational and optimal way. But how is it a problem that even ranks compared to them not addressing threats because they don’t know about them? And who are you to not tell people about genuine risks to them that they would act on, to protect their feelings or because they would be more upset than you want?
An odd aspect of discussing serious threats is the amount of concern people express about you causing other people to be concerned. This kind of makes sense for interlocutors who don’t believe in the threat itself, or think it is overblown (though in that case it is perhaps strange to focus on altruistic concern for potential frightened onlookers rather than the object-level disagreement). But often the person is not actually disputing the threat, they purportedly just want to protect the public from fear, or avoid causing ‘panic’.
A memorable case: on what was to be one of the last normal weekends in 2020, I took an Uber with a friend to an event. On the way we discussed the rising warnings of an international pandemic and our preparations. But my friend wanted us to talk more discreetly, lest we scare the Uber driver.
Why on Earth would it be bad to scare the Uber driver? My friend believed as much as I did that a real and deadly virus was spreading and there was an imminent risk of this affecting us all, including the Uber driver. Didn’t the Uber driver have an interest in knowing about it? Wasn’t it, if anything, our responsibility to tell the Uber driver? Is the concern that, as a normal person, the Uber driver is incompetent to manage himself, and will just scream and run around or buy poorly selected prepper equipment?
In conversations about AI risk, I sometimes see the same thing. Geoffrey Hinton says he thinks there’s a 10-20% chance of human extinction, and some people seem genuinely most concerned is that maybe the press didn’t add enough disclaimers about the process by which he reached that number, and the public may get unnecessarily worried. I agree it would be non-ideal if people were 20%-level worried when they would only endorse being 7% worried on further methodological inspection. But among non-ideal aspects of a situation where most of the relevant scientists believe their field is heading toward a modest-to-strong shot at killing us, it’s interesting to rate “maybe people will be too concerned” as a top concern.
What is going on? In this particular case, I could imagine behaving this way if the original communication seemed dishonest. But finding this dishonest seems similar to complaining if someone yells “fire!” that they should have yelled “I subjectively guess that it’s highly likely there’s a fire because of the smoke and flames but I’m not an expert”. And it seems like there’s something else going on with wanting that.
A related phenomenon: people casually mention ‘causing a panic’ as a thing that is assumed to be too terrible. Like, yes there’s some upside to warning people that there is a major threat to their lives that they can do something about. Maybe doing that will help stop the world from ending. But! What if they get all emotional? They may not act in the most clear-eyed and rational way. They may talk to each other in epistemically unvirtuous fashions and get even more concerned. They may buy too much toilet paper or run on a perfectly functional bank or protest for poorly designed policies.
I mean, indeed this is all worse than them addressing threats in the most rational and optimal way. But how is it a problem that even ranks compared to them not addressing threats because they don’t know about them? And who are you to not tell people about genuine risks to them that they would act on, to protect their feelings or because they would be more upset than you want?