When humanity avoids a disaster, it's usually because we have put preparations in place. To the uninformed, these seem like wastes of time—after all, nothing happened, so the threat wasn't real, right? However, when there weren't enough preparations, and calamity does occur, people can always find someone to blame for negligence.
The Preparedness Paradox is almost always a communication issue. If the public didn't think that safety measures were necessary after a disaster, then that's a clear sign that the issue isn't clear to the public. There seem to be two types of information loss that causes public misunderstanding.
Type 1: Information of what was known before a disaster
It seems like the public commonly misgauges how much information is known by authorities/experts before a disaster. Hindsight bias usually results in the public perceiving that authorities knew just as much before and after. If the public believes this, then the authorities must be either stupid or intentionally making poor decisions—both of which erode trust in experts and authorities. Governments knew little about COVID when it was new on the scene, which resulted in lots of conflicting guidelines being released as they were getting new information. If the uncertainty was more accurately conveyed during the progression of the pandemic, potentially the public would have been more receptive to later guidelines. Ensuring what information was known before a disaster can help the public make a more informed judgment on whether the right action was taken, given what was known.
Type 2: Information of what was done to avoid a disaster
For disasters that we did avoid, such as Y2K, extreme effort was put in by programmers to make sure the world's systems didn't fail. After nothing happened, the Preparedness Paradox kicked in, and some came to believe that it was just a hoax or overly exaggerated. This kind of miscommunication occurs when people don't know the amount of effort used for avoidance. Perhaps if just as much media coverage was used for showing the work being done as the fear of disaster, there would have been a much different reaction amongst the public. But because the effort was not communicated well, trust again eroded towards those who warned against Y2K.
The world, and especially in the US, has unprecedentedly low levels of trust towards authority and institutions[1]. I'm unsure if this is specifically a good or bad thing, but at least in terms of AI Safety this isn't great. We're inheriting decades of eroded trust, and the public is much less likely to trust claims outside their Overton Window (especially since most of the advocacy is coming from organizations[2]). Trust is normally rebuilt gradually, through many small disaster/response cycles that let institutions demonstrate credibility over time, but AI safety may not get enough of those reps before AGI arrives. Instead, it will likely be up to individuals rather than institutions to persuade the public about AI risk. Individuals are of course unstable, and prone to ad hominem attacks, but this still seems like a worthwhile lever to pursue. There is a large portion of Americans who are against AI altogether, but it doesn't seem like there's much in terms of bottom-up support for AI Safety. Siliconversations seems to be one of the few succeeding or doing work on this front. I'm not an expert in how to communicate abstract ideas effectively to a general population, but there could certainly be more attempts from individuals.
The public is whiplashed after years of ineffective communication. If AI is potentially going to result in disaster, we need more individuals who can communicate the risks and importance of this issue well.
From a public perspective, Anthropic and OpenAI are the only ones talking about x-risk, which makes it seem more like marketing for investors rather than genuine concern. ↩︎
When humanity avoids a disaster, it's usually because we have put preparations in place. To the uninformed, these seem like wastes of time—after all, nothing happened, so the threat wasn't real, right? However, when there weren't enough preparations, and calamity does occur, people can always find someone to blame for negligence.
The Preparedness Paradox is almost always a communication issue. If the public didn't think that safety measures were necessary after a disaster, then that's a clear sign that the issue isn't clear to the public. There seem to be two types of information loss that causes public misunderstanding.
Type 1: Information of what was known before a disaster
It seems like the public commonly misgauges how much information is known by authorities/experts before a disaster. Hindsight bias usually results in the public perceiving that authorities knew just as much before and after. If the public believes this, then the authorities must be either stupid or intentionally making poor decisions—both of which erode trust in experts and authorities. Governments knew little about COVID when it was new on the scene, which resulted in lots of conflicting guidelines being released as they were getting new information. If the uncertainty was more accurately conveyed during the progression of the pandemic, potentially the public would have been more receptive to later guidelines. Ensuring what information was known before a disaster can help the public make a more informed judgment on whether the right action was taken, given what was known.
Type 2: Information of what was done to avoid a disaster
For disasters that we did avoid, such as Y2K, extreme effort was put in by programmers to make sure the world's systems didn't fail. After nothing happened, the Preparedness Paradox kicked in, and some came to believe that it was just a hoax or overly exaggerated. This kind of miscommunication occurs when people don't know the amount of effort used for avoidance. Perhaps if just as much media coverage was used for showing the work being done as the fear of disaster, there would have been a much different reaction amongst the public. But because the effort was not communicated well, trust again eroded towards those who warned against Y2K.
The world, and especially in the US, has unprecedentedly low levels of trust towards authority and institutions[1]. I'm unsure if this is specifically a good or bad thing, but at least in terms of AI Safety this isn't great. We're inheriting decades of eroded trust, and the public is much less likely to trust claims outside their Overton Window (especially since most of the advocacy is coming from organizations[2]). Trust is normally rebuilt gradually, through many small disaster/response cycles that let institutions demonstrate credibility over time, but AI safety may not get enough of those reps before AGI arrives. Instead, it will likely be up to individuals rather than institutions to persuade the public about AI risk. Individuals are of course unstable, and prone to ad hominem attacks, but this still seems like a worthwhile lever to pursue. There is a large portion of Americans who are against AI altogether, but it doesn't seem like there's much in terms of bottom-up support for AI Safety. Siliconversations seems to be one of the few succeeding or doing work on this front. I'm not an expert in how to communicate abstract ideas effectively to a general population, but there could certainly be more attempts from individuals.
The public is whiplashed after years of ineffective communication. If AI is potentially going to result in disaster, we need more individuals who can communicate the risks and importance of this issue well.
https://www.pewresearch.org/politics/2019/07/22/trust-and-distrust-in-america/ ↩︎
From a public perspective, Anthropic and OpenAI are the only ones talking about x-risk, which makes it seem more like marketing for investors rather than genuine concern. ↩︎