You know it must be out there, but you mostly never see it.
Author's Note 1: In something like 75% of possible futures, this will be the last essay that I publish on LessWrong. Future content will be available on my substack, where I'm hoping people will be willing to chip in a little commensurate with the value of the writing, and (after a delay) on my personal site (not yet live). I decided to post this final essay here rather than silently switching over because many LessWrong readers would otherwise never find out that they could still get new Duncanthoughts elsewhere.
Author’s Note 2: This essay is not intended to be revelatory. Instead, it’s attempting to get the consequences of a few very obvious things lodged into your brain, such that they actually occur to you from time to time as opposed to occurring to you approximately never.
Most people could tell you that 17 + 26 = 43 after a few seconds of thought or figuring, and it would be silly to write an essay about 17 + 26 equaling 43 and pretend that it was somehow groundbreaking or non-obvious.
But! If the point was to get you to see the relationship between 17, 26, and 43 very, very clearly, and to remember it sufficiently well that you would reflexively think “43” any time you saw 17 and 26 together in the wild, it might be worth taking the time to go slowly and say a bunch of obvious things over and over until it started to stick.
Thanks to Karim Alaa for the concept title. If you seek tl;dr, read the outline on the left and then skip to IX.
In September of 2017, if you had asked men in the United States “what percentage of the women that you personally know have experienced sexual assault?” most of them would have likely said a fairly low number.
In October of 2017, the hashtag #MeToo went viral.
In November of 2017, if you had asked men in the United States “what percentage of the women that you personally know have experienced sexual assault?” most of them would have given a much higher number than before.
(It’s difficult, for many people, to remember that they would have said a number that we now know to be outrageously low; by default most of us tend to project our present knowledge back onto our past selves. But the #MeToo movement was sufficiently recent, and the collective shock sufficiently well-documented, that we can, with a little bit of conscientious effort, resist the mass memory rewrite. Most of us were wrong. That’s true even if you specifically were, in fact, right.)
Talking about sexual assault is not quite as taboo, in the United States, as it is in certain other cultures. There are places in the world where, if a woman is raped, she might well be murdered by her own family, or forcibly married off to the rapist, or any number of other horrible things, because the shame and stigma is so great that people will do almost anything to escape it.
(There are places in the world where, if a man is raped—what are you talking about? Men can’t be raped!)
The U.S. is not quite that bad. But nevertheless, especially prior to October of 2017, sexual assault was still a thing that you Don’t Ever Talk About At The Dinner Table, and Don’t Bring Up At Work. It wasn’t the sort of thing you spoke of in polite company—
(or even in many cases with friends and confidants, because the subject is so charged and people are deeply uncomfortable with it and there are often entanglements when both parties know the perpetrator)
—and since there was pressure to avoid discussing it, people tended not to discuss it.
(Like I said, a lot of this will be obvious.)
And because people didn’t discuss it, a lot of people (especially though not always men) were genuinely shocked at just how common, prevalent, pervasive it is, once the conversational taboo crumbled.
The dark side of this is “because sexual assault was taboo to discuss openly, many people were surprised to discover just how many of their friends and family members were victims of it.”
The Vantablack side of this is “because sexual assault was taboo to discuss openly, many people were surprised to discover just how many of their friends and family members endorse it.”
(Either by actively engaging in sexual assault themselves, or at least not finding it sufficiently objectionable enough to, y’know. Object.)
In the year 1990, if you had asked people in the United States what percentage of the population was trans, you would have generally gotten back answers like “I dunno, one in a thousand? Less??” Trans folk (as distinct from, say, cross dressers or homosexuals) were much less on the public’s radar in the year 1990; it wasn’t until 1994 that Ace Ventura: Pet Detective made over $100,000,000 by relentlessly dehumanizing a trans woman. The average person would maybe have been able to conceive of the idea that one person in their entire small town might be trans, which is a far cry from present-day estimates of “one or two people in each graduating class in a mid-sized high school.”
In 1950, if you could have even gotten away with asking the question, most people in the United States would have said that very, very few people were anything other than heterosexual. Perhaps, thinking of their confirmed bachelor uncle or the lovely old widows sharing the house down the street, they might have whispered something like “half a percent?” if they were sure that the question wasn’t a trap.
But it’s likely that even gay people in the 1950’s wouldn’t have guessed “one in every thirty adults,” which was the rate of self-report in Gallup polls in 2012, or “one in every fourteen adults” (!) which was the rate in 2022. In 1950, if you had told someone that the would likely pass within nodding distance of multiple non-straight people every single time they went to the grocery store, they would have assumed you were either crazy or slandering their hometown.
The conservative party line goes something like “see?? Once we stopped punishing people for bringing it up, it started happening more and more and now it’s everywhere!”
And this is at least somewhat true, with regards to behavior. To how many people with a given inclination express that inclination, through action (especially out in the open where everyone can see, as opposed to furtively, in private).
But there wouldn’t have needed to be so much glaring and shushing, if the proportion of the population with the inclination were vanishingly small. Societies do not typically wage intense and ongoing pressure campaigns to prevent behavior that 0.0001% of people want to engage in. For as long as there have been classrooms with thirty students in them, there have been two or three queer folk in those classrooms—they simply didn’t talk about it.
(Just like we didn’t talk about sexual assault. It’s not new. It’s just newly un-hidden.)
(transness, queerness, sexual assault)
—is social dark matter. If you’re both paying attention and also thinking carefully, you can conclude that it must be out there, and that there's probably actually quite a lot of it. But you don’t see it. You don’t hear it. You don’t interact with it.
(Unless you are a member of the relevant group, and members of that group in fact talk to each other, in which case you may have a somewhat less inaccurate picture than all of the oblivious outsiders. Many women were not shocked at the information that came out during #MeToo. Shocked that it came out, perhaps, but not shocked by the stories themselves.)
I’d like to present two laws surrounding social dark matter in this piece, and at this point we’re ready for the first one:
Straightforward, hopefully. People hide things they are encouraged to hide; people do not share things they are punished for sharing.
(Twice now, I have been told by a dear, close friend that they had developed and recovered from serious alcoholism, without me ever even knowing that they drank. Both friends formed the problem in private, and solved it in private, all without giving any outward sign.)
However, while almost anyone would nod along with the above, people do not often stick around long enough to do the final step, and apply it.
Here are some examples of things that people are (or were) encouraged to hide, and nonzero punished for sharing or discussing or admitting to (starting with ones we’ve already touched on):
… this is not an exhaustive list by any means. But it’s worth pausing for a moment, for each of these, and actually applying the Law of Prevalence to each, and realizing that there’s really quite a lot more of that going on than you think.
Your perceptions of how rare something like incest is are systematically skewed by the fact that virtually everyone engaging in it is also engaging in a fairly effortful (and largely successful) campaign to prevent you from ever finding out about it. Ditto everyone who habitually uses opium, or draws tentacle hentai to decompress after work, or thinks Jar-Jar Binks was what made The Phantom Menace a truly great movie.
The punishment our culture levied on left-handed people was fairly tame, relative to (say) the punishment our culture levies on pedophiles (even those who have never abused anyone). Yet even that punishment was enough to hide/suppress seventy-five percent of left-handedness! Three in four left-handers or would-be left-handers were deterred, in the year 1900; incidence went up from 3% to 12% once the beatings began to slow.
It’s not crazy to hazard that some of the more-strongly-stigmatized things on the list above have incidence that’s 10x or even 100x what is readily apparent, just like the number of trans folk in the population is wildly greater than the median American would have guessed in the year 1980.
There’s something of a corollary to the Law of Prevalence: when the stigma surrounding a given taboo is removed (or even just meaningfully decreased), this is almost always followed by a massive uptick in apparent prevalence.
A very straightforward example of this is Nazi and white supremacist sentiment in the United States, Canada, and Europe over the previous decade. The internet is littered with commentary from people saying things like "but where did all these Nazis even come from?"
In point of fact, most of them were already there. There has not been a 2x or 5x or 10x increase in actual total white supremacist sentiment—
(though there probably has been some modest marginal increase, as more people openly discuss the claims and tenets of white supremacy and some people become convinced of those ideas for the first time, or as the culture war has polarized people to greater and greater forgiveness of those on their side, etc.)
—instead, what happened is that there has been a sudden rise in visibility. Prior to 2015, there was a fragile ban on discussing the possible superiority or inferiority of various races, similar to the ban on discussion of sexual assault pre-#MeToo. In most venues, it simply Wasn't The Sort Of Thing One Would Mention.
And many, many people, not getting the Law of Prevalence all the way down in their bones, mistook the resulting silence for an actual absence of white supremacist sentiment. Because they never heard people openly proselytizing Nazi ideology, they assumed that practically no one sympathized with Nazi ideology.
And then one Donald J. Trump smashed into the Overton window like a wrecking ball, and all of a sudden a bunch of sentiment that had been previously shamed into private settings was on display for the whole world to see, and millions of dismayed onlookers were disoriented at the lack of universal condemnation because surely none of us think that any of this is okay??
But in fact, a rather large fraction of people do think this is okay, and have always felt this was okay, and were only holding their tongues for fear of punishment, and only adding their voices to the punishment of others to avoid being punished as a sympathizer. When the threat of punishment diminished, so too did their hesitation.
The key thing to recognize is that this is (mostly) not an uptick in actual prevalence. Changes to the memetic landscape can be quite bad in their own right—the world is somewhat worse, now that white supremacy is less afraid of the light, and discussions of it are happening out in the open where young and impressionable people can be recruited and radicalized, and the marginal racist is marginally bolder and more likely to act on their racism.
But it’s nowhere near as bad as it looks. Judging by visibility alone, it seems like white supremacy has become 10x or 20x more prevalent than it was a decade ago, but if there had actually been a 10-20x increase in the number of white supremacists, the world would look very, very different.
Or, to repurpose the words of Eugene Gendlin: “You can afford to live in the world where there are this many white supremacists and white supremacist sympathizers among your neighbors and colleagues, because (to a first approximation) you were already doing so.”
(Ditto anti-Semites, ditto misogynists, ditto pedophiles, ditto people hearing voices, ditto people committing all those perfect crimes that you’ve always been told don’t exist but if you stop and think for twelve seconds you’ll realize that they totally exist, of course they exist, they’re probably happening all the time, and yet somehow the world keeps spinning and society doesn’t collapse, it is worse than you thought but it’s not as bad as you would naively expect, DON'T PANIC.)
There is a substance called shellac that is made from the secretions of the female lac bug, formal name Kerria lacca. Out in the wild, shellac looks like this:
This substance has many uses. One such use is making the outer shells of jelly beans shiny and hard and melt-in-your-mouth soluble.
Many people, upon discovering that the jelly beans they have been enjoying all their lives are made with something secreted by a bug—
(I mean, what a viscerally evocative word! Secrete. Squelchy.)
—many people, upon discovering that the jelly beans they have been enjoying all their lives are coated in bug secretions, experience a sudden decrease in their desire to eat them.
This is the fearful swerve.
Here is (roughly) how it works:
The human mind has something like buckets, for various pieces of information, and those buckets have valences, or karma scores.
For many people, “jelly beans” live in a bucket that is Very Good! The bucket has labels like “delicious” and “yes, please!” and “mmmmmmmmmmm” and .
“Bug secretions,” on the other hand, live in a bucket that, for many people, is Very Bad. Its labels are things like “blecchh” and “gross” and “definitely do not put this inside your body in any way, shape, or form.”
When buckets collide—when things that people thought were in one bucket turn out to be in another, or when two buckets that people thought were different turn out to be the same bucket—people do not usually react with slow, thoughtful deliberation. They usually do not think to themselves “huh! Turns out that I was wrong about bug secretions being gross or unhygienic! I’ve been eating jelly beans for years and years and it’s always been delicious and rewarding—bug secretions must be good, actually, or at least they can be good!”
Instead, people tend to react with their guts. The bad labels trumps the good ones, the baby goes out with the bathwater, and they reach for a Reese's instead.
(In the case of shellac specifically, this reaction is often short-lived, and played up for comedic effect; there aren't that many people who give up on them long-term. But the toy example is still useful for understanding the general pattern.)
This same effect often plays out when humans become suddenly aware of other humans’ social dark matter. In the U.S. of the 1950’s (and, rather depressingly, in many parts of the U.S. of the 2020’s), the discovery that Person A was gay would often violently rearrange Person B’s perception of them, overriding (and often overwriting) all of Person B’s other memories and experiences and conceptions of Person A.
The same thing happens in various parts of the world when people are outed as having been raped, or having had an abortion, or having lost their faith, or having been diagnosed with schizophrenia, or having a habit of taking LSD on weekends, or having a penchant for wearing diapers, or any number of other taboo things. The stereotype of a good and upstanding person is incompatible with the stereotype of [dark matter], and rather than make a complicated and effortful update to a more nuanced stereotype, people often simply snap to “well, I guess they were Secretly Horrible All Along, all of my direct evidence to the contrary notwithstanding.”
This is unfortunately complicated by the fact that this update is sometimes actually correct, as many people discovered during #MeToo. There are, in fact, dealbreakers, and there are people who present a kind and friendly front while concealing horrific behavior, and one negative revelation can in fact be more important than everything else you knew about a person. Such occurrences are (sadly) not even particularly rare.
But! There’s a pretty damn important distinction between thoughts or inclinations, on the one hand, and actions on the other. Most of the horror that suburban parents in the 1950’s would have felt upon discovering that their child’s high school English teacher was gay would have been wrong—wrong in the sense of being incorrect, unfounded, unjustified, confused. The mere fact of being gay (whilst being otherwise well-behaved and in compliance with all social norms and standards, such that most people never even noticed) is not a major risk factor for child molestation, and is not evidence that Mr. So-and-So was actually a ticking time bomb all along and we simply never knew, thank God we got rid of him before he fiddled with somebody all of a sudden after never doing anything of the sort for thirty years.
The fearful swerve is a reflex. It’s one thing to conclude, after conscious thought and careful evaluation, that the things you’ve learned about a person mean that they are irretrievably Bad (especially if what you've learned is that they have actually taken harmful action out in the world).
It’s another thing altogether to leap to that conclusion unthinkingly, based on nothing more than stereotypes and pattern-matching. Deciding that "welp, jelly beans are off limits now, because I'm an ethical vegan and shellac is an animal by-product" is very, very different from just going “eww, gross!” and letting that impulse have final say.
When the categorical bucket “autism” was invented, it was built around people who were what we would now call profoundly or severely autistic. Many of them were nonverbal. Many of them had extreme tics. They were incapable of navigating ordinary human contexts, and required highly specialized and intensive assistance to survive even at home or in institutions.
Over time, the medical establishment began to pick up on autistic-spectrum traits in people whose support needs were not quite so great. In 1988, the movie Rain Man introduced the word “autistic” to the broader English-speaking public, and for a couple of decades Dustin Hoffman’s character Raymond was the go-to archetype of a person with autism. Yes, he was disabled, but he could talk, he could walk, he could feed himself, and not every surprising stimulus triggered a meltdown.
In 2010, as a middle school teacher, I encountered my own first bona fide, formally diagnosed autist, in the form of a student named Ethan. Ethan was fully capable of holding up his end of a completely normal conversation (though he much preferred that conversation be about Spore or Minecraft, and was not shy about saying so).
On December 19th, 2021, I ended a months-long process of discovering that I am autistic. These days, somewhere between 10% and 25% of my social circle wears that label as part of their identity.
The Law of Prevalence states that anything people have a strong incentive to hide will be many times more prevalent than it seems.
Part of what makes this possible at all is the Law of Extremity:
The prevalence of autism in 1988 was basically the same as it is today, i.e. really quite startlingly common, overall less rare than red hair. Yes, there are some reasonable hypotheses that certain environmental changes (not vaccines) are leading to an actual ground-level increase in the rate of autism, but if so, we’re talking about something like a 1.5x-3x increase, not a 10x or 20x or 50x increase.
But this claim—that something close to three percent of the population is autistic—would sound absolutely crazy to your everyday moviegoer leaving the theater in 1988 after watching Rain Man. It would sound crazy because that moviegoer’s picture of autism was rooted in an unusually extreme example of what an autistic person looks like. They would think you were claiming that one in thirty-three adults was Raymond Babbit, which is patently false.
The 1988 counterparts of [me] and [my autistic friends] and [my student Ethan] weren’t quite trying to hide their autism in the way that people actively try to hide things like psychopathy and pedophilia (though they probably were spending a ton of personal resources in an attempt to seem more normal than they were on the inside).
But our 1988 counterparts were able to fly under the radar at all because everyone else's autism-detectors were calibrated on much more extreme and pathological variants of the traits that they themselves carried.
When I wrote a public Facebook post letting my social circle know that I am autistic, a friend of mine replied with the following (abridged):
I've been uncomfortable with this label myself because this one time I went to a grocery store (to buy a bottle of a specific dish soap to make giant soap bubbles) and there was this guy meticulously re-stacking the product who I just ignored because I thought he was a store employee but then he grunted and shoved my hand away when I tried to take one. An older fellow identifying himself as the man's care taker showed up and had to explain the situation. Then I had to drive to another store to get that specific dish soap.But, anyway, my point is, neither of us are like that guy, and that's the sort of case I thought the psychiatric label was designed for.
I've been uncomfortable with this label myself because this one time I went to a grocery store (to buy a bottle of a specific dish soap to make giant soap bubbles) and there was this guy meticulously re-stacking the product who I just ignored because I thought he was a store employee but then he grunted and shoved my hand away when I tried to take one. An older fellow identifying himself as the man's care taker showed up and had to explain the situation. Then I had to drive to another store to get that specific dish soap.
But, anyway, my point is, neither of us are like that guy, and that's the sort of case I thought the psychiatric label was designed for.
To which my partner @LoganStrohl replied:
because the characteristics you're mostly likely to observe in someone are the easy-to-see-from-the-outside ones (unusual social behaviors, motor problems, speaking problems), you'll probably by default see people with extreme versions of these more visible characteristics as "more autistic".but there are autistic people who can neither walk nor talk but are unusually good *for a median human* at flirting. there are autistic people who have severe world-shattering meltdowns daily from sensory overload yet have built such convincing masking and coping mechanisms that their bosses do not know they're anything besides "a bit quirky". there are autistic people who react so violently and explosively to perceived rule violations that they need constant supervision to protect others and themselves, yet they find life in the middle New York City comfortable and exciting rather than overwhelming.i think there is a thing going on with intelligence, where people with higher intelligence tend to succeed more at going about normal life than people with lower intelligence. if two people share basically the same innate tendencies wrt information processing style, social processing architecture, sensory processing ability, etc.; but one is IQ 95 while the other is IQ 160; then the IQ 95 person may seem like a total disaster who can't dress himself let alone hold down a job, while the IQ 160 person has stable employment, a mortgage, and lots of friends who appreciate his quirky interests and moral passion. one of these people isn't more autistic than the other; they're just more *disabled* than the other.there's a very similar effect from support. imagine two very similar autistic people again. one of them is born into a wealthy progressive family, they go to the same school and live in the same town the whole time they're growing up, their parents organize a lot of structured interest-based socialization for them, and then they go to college where food is provided by resident services in the cafeteria while they study computer science until they move on to the Google campus. the other person grows up moving from foster home to foster home in a conservative area where differences are seen as threatening, and they're considered a problem child and put through ABA therapy that punishes their coping strategies and tries to cure their disorder. the first person will not stick out nearly as much to someone looking for obvious external signs of autism.given that Duncan is both very intelligent and has been very well supported in the relevant ways throughout his life, then on the hypothesis that he is autistic, we should expect his internal experiences and personal struggles to be WAY more similar to those of supermarket guy than they appear (much more similar to those of supermarket guy than to those of a randomly chosen person)
because the characteristics you're mostly likely to observe in someone are the easy-to-see-from-the-outside ones (unusual social behaviors, motor problems, speaking problems), you'll probably by default see people with extreme versions of these more visible characteristics as "more autistic".
but there are autistic people who can neither walk nor talk but are unusually good *for a median human* at flirting. there are autistic people who have severe world-shattering meltdowns daily from sensory overload yet have built such convincing masking and coping mechanisms that their bosses do not know they're anything besides "a bit quirky". there are autistic people who react so violently and explosively to perceived rule violations that they need constant supervision to protect others and themselves, yet they find life in the middle New York City comfortable and exciting rather than overwhelming.
i think there is a thing going on with intelligence, where people with higher intelligence tend to succeed more at going about normal life than people with lower intelligence. if two people share basically the same innate tendencies wrt information processing style, social processing architecture, sensory processing ability, etc.; but one is IQ 95 while the other is IQ 160; then the IQ 95 person may seem like a total disaster who can't dress himself let alone hold down a job, while the IQ 160 person has stable employment, a mortgage, and lots of friends who appreciate his quirky interests and moral passion. one of these people isn't more autistic than the other; they're just more *disabled* than the other.
there's a very similar effect from support. imagine two very similar autistic people again. one of them is born into a wealthy progressive family, they go to the same school and live in the same town the whole time they're growing up, their parents organize a lot of structured interest-based socialization for them, and then they go to college where food is provided by resident services in the cafeteria while they study computer science until they move on to the Google campus. the other person grows up moving from foster home to foster home in a conservative area where differences are seen as threatening, and they're considered a problem child and put through ABA therapy that punishes their coping strategies and tries to cure their disorder. the first person will not stick out nearly as much to someone looking for obvious external signs of autism.
given that Duncan is both very intelligent and has been very well supported in the relevant ways throughout his life, then on the hypothesis that he is autistic, we should expect his internal experiences and personal struggles to be WAY more similar to those of supermarket guy than they appear (much more similar to those of supermarket guy than to those of a randomly chosen person)
…which is in fact true; I was immediately able to recognize and describe several ways in which the supermarket guy seemed to be something like a less-well-contained version of myself.
The first autistic people to rise to the level of attention were those who:
And thus, when the stereotype (/archetype) of autism began to crystallize, it crystallized with those people at the center, when in fact they were well to the right side of the bell curve.
This (claims the Law of Extremity) is approximately always going to be true. It doesn’t matter if you’re talking about autism or homosexuality or sexual assault or drug use—when it comes to social dark matter, what you hear about on the news and see depicted in movies and shows will always be an unusually vivid version of [whatever it is]. Someone who couldn’t hold it together, couldn’t keep up the facade—someone whose [thing] is so intense, or whose self-control is so tattered, that they can’t keep it under wraps like everybody else.
A term-of-art in this domain is representativeness; there’s a difference between the most memorable/most recognizable version of a thing, and the actual most common or typical example. The classic archetype of a basketball player is a thin, muscular, dark-skinned man, pushing seven feet tall and wearing a jersey with a number on it. But the vast majority of the people actually playing basketball at any given moment do not fit this description.
You could call it the Law of Caricature, perhaps. The picture of a drug user is a lank, twitchy, pale young man with greasy hair and disheveled clothes and wild, suspicious eyes—but most drug users look nothing like that. A New Yorker cartoon depicting a Jew might have characteristic long braids and dark clothes and a yarmulke and a prayer shawl with a star of David—but most Jews do not look like that, most of the time.
(If #MeToo taught us anything, it taught us that you can't identify rapists and molesters simply by looking for people who "look like" rapists and molesters.)
Just as 95+% of people who play basketball look nothing like LeBron James, so too is it obvious (if you pause and actually take the time to add 17 to 26) that 95+% of psychopaths aren’t serial killers and 95+% of pedophiles aren’t child molesters and 95+% of drug users have never mugged a stranger on the street to pay for their next hit.
("Even if most Dark Wizards are from Slytherin, very few Slytherins are Dark Wizards. There aren't all that many Dark Wizards, so not all Slytherins can be one.")
The perception that all of these traits are necessarily extreme and dangerous is explained by the fact that the visible examples of [psychopaths] and [pedophiles] and [drug users] are all drawn from the extreme end of the distribution. When the penalty for a certain kind of dark matter is steep enough, the only individuals you notice are the ones who cannot escape notice, and if you don't keep your brain on a fairly short leash, it will naturally default to a corresponding belief that the entire population of [psychopaths] and [pedophiles] and [drug users] centers on those individuals.
But in all likelihood, for every person wandering the street raving about freemasons, there are literally dozens who hear voices from time to time and never unravel enough that they even get diagnosed.
(Or, to put it another way: you can’t confidently deduce an absence of people-hearing-voices in a given population from an absence of people raving on the street, just like you shouldn’t be confident that no one is playing basketball because none of them are seven feet tall.)
(Similarly, #MeToo showed us that it's a mistake to conclude that none of the people in our lives have been sexually assaulted simply because none of them are suffering nightmares and panic attacks and having trouble holding down a job and conspicuously avoiding being in the same room as their boss. Most rape victims don’t look anything like the training data our society handed us.)
I once got into a protracted disagreement with a colleague about the prevalence of a specific form of social dark matter. I used eight different independent lines of reasoning to try to reach an estimate of how commonplace it might be, each of which gave me back an answer somewhere in the ballpark of 0.5% - 5%.
(Several of the lines of reasoning were very crude/coarse/rudimentary, such that if any one of them had been my sole source of intuition I would have been embarrassed to proffer my guess, and might have preferred to say “I really don’t know.” But eight unrelated crappy methods all giving answers in the same order of magnitude felt like real reason for confidence.)
My colleague did not buy it. They were insistent that prevalence had to be well below one in a thousand. They were someone in whom people frequently confided, on a fairly wide range of issues—"I’ve been told equally damning things, multiple times; I find it hard to believe I would have simply never heard this particular one, from anyone, unless it is in fact vanishingly rare."
And yet I had received (and continued to receive) confidences from multiple such people—more than one of whom was also close to my colleague and yet chose not to extend their trust in that direction.
This is no accident; most people systematically drive away such confidences (and I work rather hard, and rather deliberately, to earn them). The driving-away is usually not intentional; it’s a vibe thing. It comes across in offhand comments and microexpressions and reflexive, cached responses that the speaker will hardly even notice themselves saying (because they’re so self-evidently true).
If you walk around aggressively signaling the equivalent of “What!?? There ain’t no faggots in this town,” then it should be no surprise to you that your lived experience is that none of the people around you are gay (even though base rates imply that quite a few of them are).
If you exude “we are all brothers and sisters in Christ,” then you will generally not hear from atheists (except the unusually militant and confrontational ones), and you will likely underestimate their prevalence in your religious community.
(And you may also underestimate the prevalence of nonbinary folk in your immediate vicinity! The thing that tells someone’s gut that you are not safe to confide in is often indirect or subtle, and speaking always of “brothers” and “sisters” and never of “siblings” or “children” is exactly the sort of subtle signal that puts the animal instincts inside of a marginalized person on high alert.)
I have personally been in rooms where someone was trying to make a small accommodation for [some trait X], or push for being less alienating and offputting to [Xers], and heard colleagues push back, not merely against the proposed changes, but against even discussing them, arguing "look, this is all hypothetical, it's not like any of us here are actually [X], why are we even spending time on this?" when I knew for a fact that at least one other person in the room was secretly [X].
Dwight: Okay, now who wrote this hysterical one? 'Anal fissures'?Kevin: That's a real thing.Dwight: Yeah, but no one here has it.Kevin: (softly) ...someone has it.
Dwight: Okay, now who wrote this hysterical one? 'Anal fissures'?
Kevin: That's a real thing.
Dwight: Yeah, but no one here has it.
Kevin: (softly) ...someone has it.
If you loudly assert that of course none of the people in your circle are suppressing deeply violent fantasies, all of your associates are good people—then the good people around you who have their deeply violent fantasies well in hand are unlikely to disabuse you. If you would never hear from a pedophile in the world where there are none around you, and you would also never hear from a pedophile in the world where there were secretly a dozen in your social circle, then you can't use the fact of having never heard from one to tell you which world you're in.
It's a bit dangerous, epistemically, to claim that a lack of observations of a thing constitutes evidence for that thing, but in the case of social dark matter, it really can be. There's a spectrum ranging from "I will actively harm white ravens if I see them" to "I will actively reward white ravens if I see them," and where a would-be observer sits on that spectrum strongly influences how many white ravens will allow themselves to be noticed in the first place.
"I won't actively harm white ravens, necessarily, but I react to claims of their prevalence with open incredulity and scoff at people who claim to see them regularly and outright ignore arguments that we should expect a lot of them on priors and also by the way I sometimes talk about how white ravens are kinda bad maybe, and make faces whenever they're brought up" is a lot closer to the former than the latter. You don't want to fall into conspiracy-theory type thinking, but you don't have to observe all that many white ravens surreptitiously blacking their feathers with coal dust before you learn to roll your eyes at the people running around insisting that 99.9% of all ravens are definitely black ("and thank goodness for that!").
(After all, what does one have to gain by saying something incendiary like "I was introduced to sex by an adult, as a child, and I think that this was straightforwardly good for me, actually"? The few times I have seen an otherwise-apparently-happy-and-well-adjusted adult make a credible claim along those lines, they were subsequently screamed at, excoriated, gaslit, condescended-to, told that they were a victim, told that they were stupid, told that they were sick, told that they were evil, told that they were enabling child molesters, interrogated and pressured to give up the name of whoever-it-was, kicked out of various social and professional circles, and generally treated as liars or mocked for how they are obviously in denial about their own suffering and brokenness (their stable, fulfilling adult lives and livelihoods and relationships apparently notwithstanding). And given that this is the obvious, predictable-in-advance outcome of any such disclosure, most people who might be in a position to say such a thing simply don't bother. To the extent that these people are out there, they have no incentive whatsoever to disillusion the rest of us. Which is bad news for those of us who want to understand the actual state of affairs!)
You also don't have to observe all that many furtive white ravens to develop a strong default mistrust of the things that people confidently claim to know about the properties of white ravens (since all of their conclusions are drawn from an extreme, impoverished, and wildly non-representative data set). It's like trying to learn about the effects of various drugs by studying only individuals in court-ordered rehab. The people who are most emphatic about knowing precisely how awful X is tend to be the very same ones who plug their ears and go lalalalala whenever anyone pops up with counterevidence.
(Which, again, isn't to say that X is secretly good, necessarily. Just that the current situation doesn't let you tell one way or the other. If nobody ever talked about sex at all, and so you based your entire understanding of human sexuality on the study of convicted rapists, you would likely have some pretty weird beliefs about sex-writ-large, even if all of your individual bits of evidence were true and accurate.)
Okay, that one’s not actually a law. But it’s the right prescription, in the sense of being the correct action to take a majority of the time.
Core claim: If you were a median American in the United States in the year 1950, and you discovered that your child's teacher or coach or mentor was secretly in a homosexual relationship with another adult, you would very likely have felt something like panic. The median American in that era would have concluded that the teacher was obviously a clear and present danger, and taken immediate steps to protect their child from that danger, and the resulting fallout would have likely completely derailed that person's life.
And this (I further claim) would almost certainly have been bad—a moral tragedy, a miscarriage of justice—and at the root of the badness would have been people being really really wrong, as a question of simple fact, about what was implied and entailed by the mere presence of [homosexuality].
Ditto if someone adopted the Socially Appropriate Level of Panic about, say, a female neighbor in the 1600s after spotting her stewing herbs in the moonlight.
Ditto if someone adopted the S.A.L.o.P about their neighbor in California in 1942, after discovering that they were not, as had been assumed, one-eighth Korean, but actually one-eighth Japanese.
Since our moral evolution has probably not yet brought us to a state of overall perfect judgment, it seems quite likely to me that there are dozens of other tragedies of similar form taking place right now, in today's society—conclusions being drawn, and consequences being meted out, which are overwhelmingly approved-of but which future generations will (correctly) assess as having been wildly unjustified and excessive and wrong.
(That’s the main thing I’d like to see happen less. I'm not exactly impressed with how the machine handles actual rapists and racists and serial killers and child molesters, either, but I feel much less urgent about improving the situation for those people; even in situations where their actions weren't exactly choices and they’re unfortunately subject to forces beyond their control, the majority of my sympathy and protective energy nevertheless goes to their victims and their potential future victims.)
The casus belli of this essay is "can we maybe not, then?" and this final section is about how.
Much of the subsequent advice will be incomplete, and useful only if laid atop some preexisting base of skill and motivation. I'm going to try to gesture at the shape of the problem, and in the general vicinity of (what I suspect are) the solutions, but the actual nitty-gritty is up to you, and will require nonzero mental dexterity and field-ready metacognitive competence. I’ll be recommending a class of actions that are only available to you if you are at all capable of e.g. noticing a thought while it’s still forming, or recognizing the difference between an observation and an interpretation of that observation, or pausing and counting to ten and having the fact that you paused and counted to ten actually break a runaway cascade of thoughts.
I've sketched out The Bad Thing above in brief, but to add a little more detail, here's the problem as I understand it:
It seems to me that there are two basic strategies for replacing this story with a more moral, less wrong one. One of them is sort of clunky and brute-force-y, but it has the virtue of being a thing you can do once, in the course of a single afternoon, and then be done with. The other is more integrated, and has some ongoing attentional costs, but the particular kind of attention required happens to be useful for a bunch of other things and so might be worthwhile anyway.
The brute force strategy is to do something like write down and consciously consider some ten or twenty or thirty important flavors of dark matter, and to meditate on each for five minutes and figure out your personal policy around each of them. For example:
In short: to what extent, and for what reason, do you endorse actually distrusting people who (it turns out) are carrying around vast quantities of hidden rage, or drug users whose drug use is invisible to outside observers, or perverts whose perversion has remained confined to their fantasies for decades, or people with an utter absence of empathy whose outward behavior has nevertheless passed everyone's emotional Turing tests?
(By the way, the point here is not to presuppose the answer—perhaps you do still endorse distrusting them! Different people have different thresholds for risk, after all, and different people assign different moral weights to various harmful behaviors. The thing I am advocating against is joining lynch mobs by reflex and then absolving yourself of responsibility because you were merely following the orders of your culture. It's every moral person's job to do better than what was handed to them, even if only by a little, and actually bothering to try to answer this question at all is a damn good start.)
One hugely beneficial side-effect of thinking through specific kinds of dark matter that are likely pretty prevalent in the population is that it gets the initial shock out of the way. When you believe yourself to live in a world where no one around you regularly visualizes murdering their coworkers in grim detail, it is understandably unnerving to suddenly discover that someone on your team does!
And in the resulting shock, confusion, and panic, it's easy to succumb to a generic, undirected pressure-to-act. It feels like an emergency. It feels like you're in danger. It feels like something must be done.
(And it's easy to forget the Law of Prevalence, and that probably several of your coworkers have such thoughts, and it's easy to forget the Law of Extremity, and that such thoughts probably influence and control people's actual behavior much less than media has primed you to think.)
And people who are frightened and disoriented and triggered often flail, taking actions that are desperate and disproportionate and poorly thought-out as they try to claw back some sense of safety and control.
Running through considerations like the above for even just a handful of specific strains of social dark matter does something like inoculate you against that sort of panic response, such that when you drop by your friend's house unexpectedly and discover them halfway into putting on a fursuit, you aren't wholly ad-libbing, and have some previous prep work to fall back on.
(It's sort of like how, at least in theory, having practiced kicks and punches in the context of martial arts kata means that your muscles have pretrained patterns that will fire if you end up in an actual fight. Meditating on how you would ideally want to respond to a given unmasking makes it more likely that you will respond that way, as opposed to doing something more frantic and mercurial.)
The other path to not-doing-the-thing is to catch it in the moment, i.e. to notice that your brain is going off the rails and intervene before it does anything drastic and irrevocable.
(Often, literally just noticing what's happening at all, as in, popping out of the object-level flow and thinking/saying something like "I notice I'm panicking" is enough.)
The whole problem with social dark matter is that our underlying base of experience is deeply impoverished, which means that our unexamined beliefs are largely wrong, which means that our default responses tend to be miscalibrated and broken. It's like someone who's never driven on ice before reflexively slamming on the brakes, because slamming on the brakes usually makes a car stop—your mental autopilot isn't prepared for this, and so letting your default reactions guide you is a recipe for disaster.
But if you can successfully replace even one would-be link in the chain with the action [noticing what's actually happening in your brain], the odds of snapping out of it and waking up go through the roof.
(For a dozen more essays on this, check out The Nuts and Bolts of Naturalism.)
So here, for your perusal, is a(n incomplete) list of things you might notice, if you are in the middle of something like a fearful swerve—thoughts or feelings or sensations that could be a clue that you want to do something like slow down and start reasoning more explicitly.
All of these things tend to push people forward along the default track. They're the opposite of the standard, virtuous, building-block move of rationality: "wait, hang on a second. What do I think I know, and why do I think I know it?"
And if you manage to become cognizant of any of those things happening in your brain, then you've already created space for yourself to simply not take the next step. To break the cascade, and instead take any number of options that buy you thirty seconds of additional breathing room.
(Counting slowly to ten, taking a dozen deep breaths, saying out loud "pause—I need to think," getting up and going to the bathroom, remembering your feet.)
The point is not so much to do some specific wiser move as it is to simply avoid doing the known bad thing, of blindly following your broken preconceptions to their obvious and tragic conclusion.
(Remember: if, after thirty seconds of conscious awareness and deliberate thought, you come to the conclusion "no, this actually is bad, I should be on the warpath," you can always ramp right back up again! Any panic that can be destroyed by a mere thirty seconds of slow, deep breathing is probably panic you didn't want in the first place, and it's pretty rare that literally immediate action is genuinely called-for, such that you can't afford to take the thirty seconds.)
What is social dark matter?
It's anything that people are strongly incentivized to hide from each other, and which therefore appears to be rare. And given that our society disapproves-of and disincentivizes a wide variety of things, there is a lot of it out there.
By the Law of Prevalence, any given type of social dark matter is going to be much more common than your evidence would suggest, and by the Law of Extremity, instances of that dark matter are going to tend to be much less extreme than you would naively guess.
If you lose track of these facts, you can go crazy. You can end up thinking absolutely insane things like "all psychopaths are violent, manipulative bastards" or "pedophilia is an absolute, irresistible compulsion" or "none of the women I know have ever been sexually assaulted."
You can end up thinking these things because your mind takes, as its central, archetypal examples of a thing, what are actually the most extreme, most visible, most unable-to-be-hidden instances of it.
As a result, when you glimpse something through a window or someone offers you a confidence or you stumble across an open journal page, you are in danger of making some pretty serious mistakes, chief among them concluding that this real person is well-modeled by your preexisting stereotype.
(This is called "bigotry"—reducing someone to a single demographic trait and then judging them based on your beliefs about that trait.)
And as a result of that, there are frighteningly high odds that you will fly off the handle in some fashion, and punish them for things they not only haven't done, but also very well might never have done in the future, either.
And that's kind of a bummer, and maybe something that you don't want to do, on reflection! Fortunately, you have other options, such as "think through such situations in advance" and "develop the general ability to notice your brain going off the rails, and when you do, maybe just ... slow down for a bit?"
I will reiterate that I think we have different moral responsibilities toward people with dark matter traits, versus toward people with dark matter actions. The advice "think it through in advance" and "slow down for a bit in the moment" applies equally well to discovering that your partner fantasizes about violently raping people as it does to discovering that your partner has violently raped someone, but in the latter case, the correct thing to do is (pretty often, at least) to out and report them. The reason you don't necessarily out the person in the former case is because our society is systematically confused about how rare and how damning the existence of a rape fetish really is—
(largely by virtue of making it extraordinarily difficult to go gather the data, although some heroes are trying anyway)
—and so you're causing many of their future interactions with employers and friends and potential romantic prospects to be unfairly dominated by this thing that may actually have very little to do with their core character and their actual behavior.
And that's bad! It's bad to punish the innocent, and when it happens, it just further reinforces the problem, making other innocents with the same traits hide even harder in a spiral of silence.
If you would like to think of yourself as the sort of person who would hide Jews from the Nazis, because the things that the Nazis believed about Jews being fundamentally and irretrievably toxic and horrible and corrosive and evil were simply wrong—
If you would like to think of yourself as the sort of person who wouldn't have destroyed the career of a high-school English teacher in the 1950's, because the things that society believed about gay men being universally corrupting sex-craved child molesters were simply wrong—
If you would like to think of yourself as the sort of person who wouldn't have burned witches, or ostracized prostitutes, or stigmatized the mentally ill, or fought to maintain segregation, then you should apply the same kind of higher perspective now, and see if you can sniff out the places where the current social consensus is malignantly confused, just as the Jew-hiders and slave-rescuers and queer allies of the past saw past their societies' unquestioned beliefs.
My parting advice, in that endeavor: take seriously the probability that you have not, in fact, internalized this lesson. This essay, by itself, was probably not enough to get you to really truly grok just how many sadists and coke users and schizophrenics and brotherfuckers and closeted Randian conservatives and Nickelback fans you have in your very own social circle.
They were always there, and (in many cases) things turned out okay anyway. When you have an unpleasant revelation and it feels like the sky is falling, check and see whether there's an actual problem that requires you to take action, or whether it's just your preconceptions shattering.
Much of the time, it's the latter, and acting anyway is what leads to tragedy.
Part of what gives me confidence to name the Law of Prevalence the Law of Prevalence, rather than the Guess of Prevalence or the Maybe of Prevalence, is that this happens basically every single time a given stigma is reduced.
"What is true is already so.Owning up to it doesn't make it worse.Not being open about it doesn't make it go away.And because it's true, it is what is there to be interacted with.Anything untrue isn't there to be lived.People can stand what is true,for they are already enduring it."
With a special exception for honey, somehow; never underestimate the power of normalization.
When a single revelation triggers a massive update, it's usually (in my experience) because one's model of the other person ruled out that they would ever do X, and so discovering even a single instance of X is a Big Deal because it means that one's model of X being firmly off limits was fundamentally flawed.
It's a reflex people do at themselves, too. Social dark matter is often dark to the person possessing it; people often dread finding out or admitting (even just to themselves) that they are queer or lack empathy or are turned on by violence or were sexually assaulted, etc.
Or perhaps highly disabled people with autism; sorry, I’m not sure where we currently stand with respect to which terminology manages to be neutral.
Or someone who chooses not to, for various reasons, but when the incentives are steep, few people make that choice and so you're still looking at someone who is Notably Nonstandard.
I recall one of my partners (who worked as a stripper) mentioning that their father remarked "strip clubs are where dreams go to die," to which my partner's knee-jerk response was "what? No—strip clubs are where pre-meds go to pay off their student loans."
They also, for the record, did not rebut or even substantively engage with any of my eight distinct lines of reasoning. They ignored that part of my argument completely, as if I hadn't said anything at all.
There are child molesters out there! Extremely distressingly many! Something like ten percent of all minors are sexually abused, and often by teachers and pastors and scoutmasters and other Trusted Authority Figures! It would be great if we could figure out how to actually preemptively identify such people. But "they're gay" just ... turned out not to be a marker, in the way that people in the 1950's wanted it to be. And being overconfident in your belief that you've found a proxy or correlate, when you haven't, makes things worse for victims. It makes you wrongly attack trustworthy and innocent people who have the trait, and it makes you wrongly trust dangerous people who lack it.
Some various examples: they will betray the United States to Japan, or brainwash your children into thinking that they should cut off their own genitals, or steal your spouse's valuables and sell them to a pawnshop to fuel their addiction, or try to play doctor with an unsuspecting child, or lose control in a traffic jam and grab their gun and kill someone.
While in the process of writing this very essay, I made a Facebook post in which I went out of my way to be careful with the label "psychopath," and this triggered the following real-world example of the thing, happening in real time:
I am stuck at the idea that it's bigoted to tell someone not to date or get close to psychopaths. Like, dude, would it be bigoted to say "don't marry a woman who will baby trap you, or one who will immediately run away with your baby"? Like, these are obvious no goes in the mating pool, and all parents should tell their kids. "Once upon a time, we were in Egypt. Oh, and also avoid psychopaths. People who take pleasure in hurting others make bad social connections on any level."Also, where on the surface of the earth can you be if not engaging with psychopaths is a difficult problem? 'I would lose so many potential social connections by filtering out (checks notes) enjoys hurting people.' Jeez, do you work for the police or for a bank? Are your coworkers doing lines of coke in the bathroom or straight up at their office desk?
I am stuck at the idea that it's bigoted to tell someone not to date or get close to psychopaths. Like, dude, would it be bigoted to say "don't marry a woman who will baby trap you, or one who will immediately run away with your baby"? Like, these are obvious no goes in the mating pool, and all parents should tell their kids. "Once upon a time, we were in Egypt. Oh, and also avoid psychopaths. People who take pleasure in hurting others make bad social connections on any level."
Also, where on the surface of the earth can you be if not engaging with psychopaths is a difficult problem? 'I would lose so many potential social connections by filtering out (checks notes) enjoys hurting people.' Jeez, do you work for the police or for a bank? Are your coworkers doing lines of coke in the bathroom or straight up at their office desk?
The person replying to me seems unaware that they are making a leap from "psychopath" to "liar/manipulator/sadist." They don't have distinct buckets for separately assessing [A], [B], and [A → B]. The archetype is the whole bell curve, as far as they know. It's analogous to the (also often completely elided) differences between [pedophiles] and [child molesters], or between [people with violent sexual fantasies] and [rapists].
This is true when reacting to epiphanies about your own dark matter, too; it's likely that at least some queer suicides are not just about predictions of how others will treat them, but about a desperate attempt to solve the problem from the inside. Please just breathe.
With the Different Worlds caveat that bubbles do exist and there probably is somebody out there for whom this statement is actually true. But you can't be (responsibly) confident in such a belief simply because the women around you never talk about it.
I will note that one of my beta readers helpfully provided the psychopathic perspective that these empathetic and ethical perspectives are not particularly motivating. I think that's a solid point! I would bet on the existence of a selfish, purely pragmatic argument for these actions, but I have not actually gone out and located it. A short gesture in the direction of my intuition is "on long enough timelines and with broad enough horizons, the actions taken by an MTG black agent often converge with those taken by an MTG white one." (There's also a purely pragmatic argument for going with the flow and not rocking the boat, though, and I ruefully acknowledge that it may be correct, according to their values, for most of my psychopathic readers to shrug and not help with this one.)
This is not to say that the situation is always good or even fine. As #MeToo demonstrated, often things are very, very bad, and society just ... ignores it, and keeps on churning. Often it would be nice if the world were more derailed by ongoing atrocity, but whether the dark matter is "good" or "bad," either way we can't get to a place of commensurate, appropriate reaction until we can manage to look straight at reality without distortion and talk clearly and frankly about what's actually going on.
Law of Extremity does some weirder stuff...
Consider a gaussian trait with high levels stigmatized. From a careless observer's perspective, +1s will be rare (somewhat hiding), +2s extremely rare (thoroughly hiding), +3s nonexistant (very hiding *and* rare to begin with) but +4s unable to hide and in fact talked about incessantly on the news / clickbait / juicy rumor mill. Which looks like there are two populations, a large left-skewed one and an entirely distinct but much smaller one. The trait at [-3,+1] and at +4 may not even look that qualitatively similar! So our observer uses a categorical model.
It's the wrong model. The trait is gaussian. That's the scenario.
So we have a category for people who are high in our trait with neither natural place nor social consensus on where draw a border line. This will be bad for all discourse on the subject.
Societies do not typically wage intense and ongoing pressure campaigns to prevent behavior that 0.0001% of people want to engage in.
The first time I read this I stared at it and went "wait, here's an obvious counterexample. It's a thing that basically nobody wants to engage in, but there's a pretty intense social pressure not to- oh wait."
I'm interested in how one might ever become justifiably confident a particular piece of dark matter really doesn't exist or is as rare as you'd suspect it is. I'm fairly confident there aren't shapeshifting lizardmen hiding among my friends for instance. I want there to be a way to trade action for knowledge- to credibly claim I won't get upset or tell anyone if a lizardman admits their secret to me- but obviously the lizardman wouldn't know that I could be trusted to keep to that, and anyway it is sometimes preferable to know without making that commitment.
I want there to be a way to trade action for knowledge- to credibly claim I won't get upset or tell anyone if a lizardman admits their secret to me- but obviously the lizardman wouldn't know that I could be trusted to keep to that,
The thing people are generally trying to avoid, when hiding their socially disapproved of traits, isn't so much "People are going to see me for what I am", but that they won't.
Imagine you and your wife are into BDSM, and it's a completely healthy and consensual thing -- at least, so far as you see. Then imagine your aunt says "You can tell me if you're one of those BDSM perverts. I won't tell anybody, nor will I get upset if you're that degenerate". You're still probably not going to be inclined to tell her, because even if she's telling the truth about what she won't do, she's still telling you that she's already written the bottom line that BDSM folks are "degenerate perverts". She's still going to see you differently, and she's still shown that her stance gives her no room for understanding what you do or why, so her input -- hostile or not -- cannot be of use.
In contrast, imagine your other aunt tells you about how her friends ... (read more)
So, to put it shortly, there are more of them, but most of them are not what you think.
This feels a bit confusing. I mean, if they are not what I think they are, could it mean that we are kinda talking about two different things? There are many "people on autistic spectrum". There are few "rain men". It's not that the former statement is more true than the latter; they are statements about different things. Are we actually debating the territory here, or just disagreeing about what the label "autist" should refer to?
(Is the conclusion I am supposed to take here something like: Many people are pedophiles, but most pedophiles actually don't want to have sex with kids... they just think that kids are kinda cute? Many people are psychopaths, but most psychopaths do not lack empathy... they just disagree with some effective altruist ideas?)
In case of autism, the statement about territory is that sometimes the things you assumed to be binary or at least bimodal are actually on a continuum ranging from normal to the extreme. And therefore... you should not have a word for "people at the extreme end of this continuum"? Words exist for a reason; apparently someone finds it useful to talk ab... (read more)
I definitely read all examples as "both at the same time."1) Whatever X publicly condemned thing you can think of, it exists on a spectrum. 2) There is a lot more of all instances of it happening than you think there are.3) A lot of it does not look like the kind you are most likely to notice and condemn.
I agree. Let me elaborate, hopefully clarifying the post to Viliam (and others).
Regarding the basics of rationality, there's this cluster of concepts that includes "think in distributions, not binary categories", "Distributions Are Wide, wider than you think", selection effects, unrepresentative data, filter bubbles and so on. This cluster is clearly present in the essay. (There are other such clusters present as well - perhaps something about incentive structures? - but I can't name them as well.)
Hence, my reaction reading this essay was "Wow, what a sick combo!"
You have these dozens of basic concepts, then you combine them in the right way, and bam, you get Social Dark Matter.
Sure, yes, really the thing here is many smaller things in disguise - but those smaller basic things are not the point. The point is the combination!
It’s hard to describe (especially in lay terms) the experience of reading through (and finally absorbing) the sections of this paper one by one; the best analogy I can come up with would be watching an expert video game player nimbly navigate his or her way through increasingly difficult levels of some video game, with the end of each level (or section) culminat
Many people are psychopaths, but most psychopaths do not lack empathy... they just disagree with some effective altruist ideas?
Lacking affective empathy is arguably one of the most defining characteristics of psychopathy, so I don't think this is a good example.
Instead, think of all the peripheral things that people associate with psychopathy and that probably do correlate with it (and often serve as particularly salient examples or cause people to get "unmasked"), but may not always go together. Take those away.
I like this post, but I think that it underestimates a bit the strength of filter bubbles. Even if a trait is present in 1% of the general population, it doesn't automatically follow that literally everyone has someone in their social circle with that trait.
I consider myself liberal enough to not project an anti-trans aura, but I still never meet trans people in person, like Scott Alexander never meet conservatives. Base rates seem to imply that there should be dozens of trans people in my town, but I've never seen one, and I don't know of anyone who has. My best guess is that when they decide to change sex, they also relocate to a more queer-friendly place. Taking SF as an extreme case, in the Bay you can find companies entirely run by trans women, a thing that I'm pretty sure never happens in other towns.
Sorry for a minor nitpick, it's worth making. It doesn't detract from Duncan's overall point at all, and if people lose Bayes points from every single nitpick, then Duncan nonetheless loses very few from this.
Because they never heard people openly proselytizing Nazi ideology, they assumed that practically no one sympathized with Nazi ideology.And then one Donald J. Trump smashed into the Overton window like a wrecking ball, and all of a sudden a bunch of sentiment that had been previously shamed into private settings was on display for the whole world to see, and millions of dismayed onlookers were disoriented at the lack of universal condemnation because surely none of us think that any of this is okay??But in fact, a rather large fraction of people do think this is okay, and have always felt this was okay, and were only holding their tongues for fear of punishment, and only adding their voices to the punishment of others to avoid being punished as a sympathizer. When the threat of punishment diminished, so too did their hesitation.
Because they never heard people openly proselytizing Nazi ideology, they assumed that practically no one sympathized with Nazi ideology.
In the case of political ideologies, it might be hard to create potential energy for intense political views (or transformative technology could sudden... (read more)
There is social pressure to hide X. So, X turns out to be much more common and much less extreme than one naively imagined. The net effect is already out there, so maybe just chill.
But, the above story exists in equilibrium with the reverse reaction:
There is social pressure to hide X. It turns out that X is much more common than one naively imagined, and although the average instance of X is not so extreme, the system is actually about to collapse under the cumulative weight of X, and almost nobody is aware until it happens.
A story like this is revealed at the end of every business cycle, where X is some form of corruption we previously thought was held in check by the social pressure against X (which turned out to be insufficiently harsh). Like, for example, approving loans to people we know are unlikely to pay them back.
The mere fact of being gay (whilst being otherwise well-behaved and in compliance with all social norms and standards, such that most people never even noticed) is not a major risk factor for child molestation, and is not evidence that Mr. So-and-So was actually a ticking time bomb all along and we simply never knew, thank God we got rid of him before he fiddled with somebody all of a sudden after never doing anything of the sort for thirty years.
I think... finding out (in the 1950s) that someone maintained many secret homosexual relationships for many years is actually a signal the person is fairly devious, and is both willing and capable of behaving in ways that society has strong norms about not doing.
It obviously isn't true about homosexuals once the norm was lifted, but my guess is that it was at the time accurate to make a directional bayesian update that the person had behaved in actually bad and devious ways.
Edit: From looking through some of a YouTube documentary linked below, I updated that many of these people seemed pretty harmless and kindly. So I think there's a good chance I'm wrong in this case.
but my guess is that it was at the time accurate to make a directional bayesian update that the person had behaved in actually bad and devious ways.
I think this is technically true, but the wrong framing, or rather is leaving out another possibility: that such a person is someone who is more likely to follow their heart and do what they think is right, even when society disagrees. This could include doing things that are bad, but it could also include things which are actually really good, since society has been wrong a lot of the time.
I think that there isn't just one bag of people breaking rules, and some number of the marbles in that bag are "for good reasons" and some number "for bad reasons." I think there are clusters, and types, and certain kinds of rule-breaking are predictive of other kinds of rule-breaking.
I think that me not wearing shoes at university is evidence that I might also disdain sports, but not evidence that I might steal.
I think that trying to think in terms of "for bad reasons" and "for good reasons" as two flavors in one bucket is likely to lead one to make wrong updates. Like, the model is oversimplified and causes fearful swerves.
Hmmmm, I see that Ben is getting some disagreement/pushback below, but I want to stand in defense of part of the thing I understand him to be saying.
(Or rather, I think Ben's bucketing two things and the disagreement is pushing back on the whole bucket when it should be pushing back on the bucket error + one of the things.)
In my culture, we're much better at noticing that X is bad in an absolute sense even if it's overwhelmingly good in context, or net justified, or whatever. Like, in Duncan culture it's straightforwardly obvious and commonplace to note that surgery is cutting into people with knives and lasers, and that kinda sucks, regardless of the fact that surgery can lead to miraculously positive outcomes.
I think that Ben is correct to point out something like "this is an update on deviousness," and I think Ben is correct to point out something like "this is an update on being willing to cross social barriers."
But I think it's not correct to leap from that to "this is an update on a fundamental or intrinsic deviousness characteristic." The deviousness of someone tying themselves in knots to avoid social sanction for behavior that's actually harmless is not quiiiii... (read more)
Liking the score from the movie "Titanic" as a seventh-grade boy in N.C in 1998.
(I don't know why, but I enjoy reading Duncan quietly slipping in references to situations that bothered him 25 years ago, all the more-so when there's vanishingly little chance that anyone involved will ever read him mention it.)
I enjoyed how raw this was. I once cried at the episode of pokemon where meowth and pikachu bond while being stuck up a tree and my brother called me gay. There was about a 5-10 year stretch where I didn't cry at all (not just linked to that). Humans can get on each other about the weirdest stuff in harmful ways.
In something like 75% of possible futures, this will be the last essay that I publish on LessWrong. Future content will be available on my substack, where I'm hoping people will be willing to chip in a little commensurate with the value of the writing, and (after a delay) on my personal site (not yet live).
Are you willing to take commissions? If I was a software engineer instead of doing AI governance in DC, I would feel like paying you $5000 every now and then to write on a topic that we're both interested in but I'm more interested in (like what would happen to the world if a critical mass of million people read WWOTF or HPMOR or dath ilan/duncanverse or signed up for cryopreservation) and post it on Lesswrong. I think that's sufficiently weird enough to be one of the 25% possible futures where you sometimes post on LW again.
Maybe lightcone could set up some kind of regranting system where they get a 10% cut for matching wealthy earn-to-givers with less-wealthy brilliant writers ("This post was funded by Sam Samson"), seems like a potential way to get more Scott Alexanders and Zvis. I have no idea how that kind of thing works though.
Yes, I have taken commissions in the past (the CFAR handbook rewrite being one, paid for by Lightcone, and Staying Split being another, paid for by an individual).
The Law of Extremity seems to work against the Law of Maybe Calm The Fuck Down. If the median X isn't worth worrying about, but most Xs you see are selected for being so extreme they can't hide, then the fact you are seeing an X is evidence about its extremity and you should only calm down if an unusually extreme X is not worth worrying about.
Ah, thanks for pointing this out. There's an unstated assumption: you stumbled across some dark matter, that was basically hidden.
If you have a full-on psychotic break, you're likely going to resemble the caricatured stereotype of a schizophrenic, and get noticed. But that's not quite the thing I'm trying to gesture at in the OP.
If somebody overhears you talking to your voices in the shower, the voices you've been talking to for decades while remaining a high-functioning individual, they're likely to leap to the conclusion that, since you have SYMPTOM of SCHIZOPHRENIA, you closely resemble the caricatured stereotype.
(This is the fearful swerve—people overupdating and dumping all of their other relevant experiential data of you, because the dark matter label overwhelms them and they forget that their new hypothesis needs to explain all the previous data too.)
The point being something like "if all that you have learned is that person has [trait], when you thought they didn't, you might be in danger of jumping to an unjustified conclusion (and creating a tragedy, as a result)."
I think my default response when I learn about [trait X] is almost the opposite of how it is described in the post, at least if I learn that someone I know has it.My mind reflexively tries to explain how [trait X] is not that bad, or good in the certain context. I have had to force myself to not automatically defend it in my head. I might signal (consciously or unconsciously) dislike for the trait in general, but not when I am confronted with someone I know having it. There are probably exceptions to this though, maybe for more extreme traits. I hope I wouldn't automatically try do internally defend rape for example, even if it was reflexive and only for one or two seconds.I just wanted to note that people like me exist too, and in certain cultures it might be fairly common (though I'm just speculating here).
Curated. This is an interesting, thoughtful, and very engagingly written discussion of the things people are incentivized to hide, and how to reason about them.
I would have considered this for curation, but I slightly more wanted to curate it given that it's Duncan's last post on LessWrong (which he assigns 75% to). I hope he continues to write many excellent essays somewhere on the public internet, and personally I have signed up as a paying subscriber to his substack to support him doing so there.
This is interesting. I hope it's true. I'm not certain in general that, if I succ... (read more)
I don't think it's precisely true. The serene antagonism that comes from having examined something and recognizing that it is worth taking your effort to destroy is different from the hot rage of offense. But of the two, I expect antagonism to be more effective in the long term.
I really wish you'd keep posting your work on LessWrong. It's much harder to figure out what's worth reading when good work on rationalism is scattered across multiple blogs. And having an active community to discuss important posts really adds to their use. Sure, there will be a few gadflies, but just not responding to trollish comments works just fine in most cases.
This is great, and several of your other pieces I've seen are similarly insightful and well communicated, on important topics. But I'm not likely to regularly check your blog. We're busy, and LessWrong has enough good rationalist content to fill the available time. I really wish your excellent content would remain part of this community.
The stereotype of a good and upstanding person is incompatible with the stereotype of [dark matter], and rather than make a complicated and effortful update to a more nuanced stereotype, people often simply snap to “well, I guess they were Secretly Horrible All Along, all of my direct evidence to the contrary notwithstanding.”
The stereotype of a good and upstanding person is incompatible with the stereotype of [dark matter], and rather than make a complicated and effortful update to a more nuanced stereotype, people often simply snap to “well, I guess they were Secretly Horrible All Along, all of my direct evidence to the contrary notwithstanding.”
Maybe people really do change their assessments of people they know well. But maybe they decide that they're not willing to take the punishment risk from appearing to defend (or even conceal) One Of Them. The best way to avoid that... (read more)
This is well put.
In particular, I liked noting that once something goes from unacceptable to discussable the previous definitions likely no longer work.
Also that if we want to be good friends we should let people show really difficult parts of themselves. Because otherwise they won't.
I think this post is probably correct, but I think most of the discourse over-complicated what I interpret to be the two core observations:
All else follows. The key takeaway for me is that you should also condition your posteriors on societal norms.
I suggest thinking about other possible social dark matter.I think deliberate weight loss makes a lot of people's lives worse-- that being hungry and distracted (possibly chilled and more frequently sick) isn't worth greater social acceptance, and that the current insistence on leanness is about looking right rather than health.Asexuality could have fit in the article.
Sorry, but in what possible sense is 'white people discussing their superiority to non-whites' part of the overton window, especially in a way that it wasn't before 2016? In what sphere is is remotely acceptable now where it wasn't before?
Literally the exact opposite is true. Intelligence researchers who research and report on the fact that black Americans have lower average IQ than white Americans and who do not dogmatically assume that it results from n... (read more)
There is such a thing as a symptom pool. Some people who are having a difficult time instinctively acquire symptoms that are fashionable and signal their difficulty, but the symptoms are not related to the actual problem. The prototypical example for this, IIRC, is incidence of anorexia in some specific country (Hong Kong?). An anorexia awareness campaign caused that symptom to become fashionable and increased the apparent incidence much more than can be explained as an increased ability to observe the cases that were present before.
This applies to some b... (read more)
Great article! The laws you describe are definitely subject to many social behaviors.
One thing I found insufficiently covered in the article is how social prejudices shape the extend people act in “dark matter” behavioral patterns. The way it’s described in the article, it seems like either Person A has property X, or they do not have property X, independent of the existing social prejudices. I would like to challenge that assumption.
If I lived many years ago, I might’ve had slaves, and could be put in a bucket “people who enjoy having slaves”. But since s... (read more)
So, I am told there are a very large number of people who hear voices, but do not otherwise have any of the seriously disabling symptoms of schizophrenia.
Possibilities that come to mind:
(in the case of autism, the argument that ... (read more)
bug secretions must be good, actually, or at least they can be good!”
So you're the same person who wrote that Magic: The Gathering Colour Wheel article. I really loved it and I thought about it a lot (then I mostly forgot about it until today).
For your next Substack post, maybe you should just post links to the best stuff you've written, so new readers know about it.
One thing that would help me, not sure if others agree -- would be some more concrete predictions. I think the historical examples of autism and being gay make sense, but are quite normalized now, that one can almost say, "That was previous generations. We are open minded and rational now". What are some new applications of this logic, that would surprise us? Are these omitted due to some info hazard? Surely we can find some that are not. I am honestly having a hard time coming up with them myself, but here goes:
It's less "you probably know a burglar" and more "successful burglaries are probably 10-100x more common than you would think, if you based your prediction solely on visible evidence."
The two that seem most obvious to me: well-behaved psychopaths (i.e. people who have little or no empathetic response but who have learned to follow the social rules anyway, for the sake of headache avoidance) and non-practicing pedophiles (i.e. people who are attracted to children but are zero percent interested in raping anyone) probably really actually are quite common.
Much like actual dark matter, "social dark matter" reveals the equation has somehow been fudged: "it must be there, because if it's not, we're wrong" is never a solid line of reasoning.In this case I think the manner in which humans and human societies operate is being radically misrepresented. A top down power analysis can easily show us that, for example, more people self identify as gay and trans in the current year vs half a century ago, because there are more political and social incentives to do so now vs half a century ago.Seeking these hidden pocke... (read more)