Paraphrasing of some of the things Jessica says:
- Bostrom's original definition of "infohazard" covers scenarios of the form "someone would like to suppress this information" in addition to "if you learn this information it will harm you".
- Scenarios where "someone would like to suppress this information" are much more common than "self-infohazard" scenarios.
- When someone wants to suppress information (such as a cult leader who's secretly abusive), it is in their interest to make others believe that it's a "self-infohazard" (including with arguments like "They're not ready to learn how I've been abusing them; it would shatter their worldview"), or possibly a "socio-infohazard" (where it's in people's collective interests to not know the leader's crimes—"It would fracture the community", "Having everyone listen to me brings order and unity and you don't want to ruin that", etc.).
I would add that it's probably best if our vocabulary choices don't make it easy for bad actors to make superficially-plausible claims that suppressing the information they want suppressed is good and virtuous. I would say that the word "infohazard" itself sounds to me (and, I suspect, to the naive layperson) like the "directly hazardous to me if I learn it" sense.
Therefore, it's probably a bad thing if serious people believe that "infohazard" technically means Bostrom's maximally-broad original definition, because that makes it harder for serious people to protest when e.g. someone says "The fact that some people in our organization screwed up X is an infohazard, so we shouldn't publicly mention it". Accepting that definition is like creating a motte-and-bailey term. (Incidentally: Learning that someone has done something bad is usually at least slightly unpleasant—more so if it's a partner or leader—and therefore it's ~always possible to make at least a slight case that "knowledge of my bad behavior" is a self-infohazard.)
I would suggest the following rule as an antidote: Anytime someone says the unqualified term "infohazard", it means "information that I want suppressed"—i.e. we should be suspicious of their motives. If they have good reasons to suppress information, they should have to state them upfront. More specific terms might be "social infohazard", "existential infohazard", "cognitive infohazard", etc. I'll also note Eliezer's suggestion of "secret":
We already have a word for information that agent A would rather have B not know, because B's knowledge of it benefits B but harms A; that word is 'secret'.
By the way, I'm kind of weirded out by the idea that we need short terms that mean "information that should be suppressed" (i.e. in terms of Huffman coding, "we should use a short word for it" = "it is very common"), and furthermore that it's rationalists who are trying to come up with such words. I think it's ultimately for innocent reasons—that Bostrom picked "information hazards" for the title of his paper, and people made the obvious catchy portmanteau; still, I don't want to push the language in that direction.
1
Suppose there were a cheap way to make nuclear weapons out of common household materials. Knowing this information is not directly harmful to the median individual like yourself. On average, it may even be beneficial to the average person, to know that secret yourself; either because you could directly conquer places, or because you could sell the info to somebody who could. But society as a whole would rather that nobody knew it than that everybody knew it.
For a more realistic example, consider the DNA sequence for smallpox: I'd definitely rather that nobody knew it, than that people could with some effort find out, even though I don't expect being exposed to that quoted DNA information to harm my own thought processes.
'Infohazard', I think, should be used only to refer to the slightly stranger case of information such that the individual themselves would rather not hear it and would say, "Don't tell me that".
A spoiler for a book you're in the middle of reading is the classic example of information which is anomalously harmful to you personally. You might pay a penny not to hear it, if the ordinary course of events would expose you to it. People enclose info about the book's ending inside spoilers, if on Reddit or Discord, not because they mean to keep that secret to benefit themselves and harm you - as is the more usual case of secrecy, in human interactions - but because they're trying to protect you from info that would harm you to know.
The term 'infohazard' has taken on some strange and mystical connotations by being used to refer to 'individually hazardous info'. I am worried about using the same word to also refer to information being alleged to be dangerous in the much more mundane sense of 'collectively destructive info' - info that helps the average individual hearer, but has net negative externalities if lots of people know it.
This was originally posted by me to Facebook at greater length (linkpost goes to original essay). Best suggestions there (imo) were 'malinfo' and 'sociohazard', and if I had to pick one of those I'd pick 'sociohazard'.
EDIT: I like shminux's 'outfohazard' even better.
EDIT 2: My mind seems to automatically edit this to 'exfohazard' so I guess that's what the word should be.