It's an interesting pattern, and the examples are fun to think about. I can produce explanations for a few categories. A very general statement, covering most of your examples, is, "To a naive person, people who don't share your goals or your background may react to your policies in ways you didn't expect or even imagine", which is downstream of the typical mind fallacy and other common mistakes.
A subcategory is "If people really want to do a thing (buy some good or service at market price; know an interesting fact; avoid doing something unpleasant; etc.), then any attempted suppression creates strong incentives to work around it, which often has fascinating results". As shminux mentions, perverse incentives can happen, and the Streisand effect actually goes somewhat beyond it (there I would say your intervention hits a preexisting immune system—people try to avoid being tricked, and either instinct or habit tells them that someone trying to hide a fact from you is a sign they're trying to trick you).
Another very general explanation: "If a complex system is already somewhat optimized in one direction, then there are a lot more ways to make it worse than to make it better, and therefore our priors on a naive attempt to 'improve' it should be pretty bad." Chesterton's fence is related.
That said, I would definitely not call this "enantiodromia" a "principle"—to me that implies it's a scientific or mathematical fact, and furthermore suggests that there's some mysterious force that specifically causes it. No: There are mundane explanations for everything, and there are heuristics you can learn (and should learn, especially if your career has any chance of affecting serious policy decisions), probably most importantly "Having thought of your proposal, try imagining the ways the different involved parties might react, and imagine ways it could go wrong". There are forces opposing the proposals, but they're different in each case (the people who just want good lawyers; the people who abuse celibacy norms; the virus researchers), and there is no mysterious force causing all of them (the thing that seems most like it is the "negative prior on naive policy proposals").
It reminds me of the concept of "synchronicity", the idea that you see more coincidences than appear to be explained by causality or by chance. As far as I can tell, the explanation for that is simply (a) sometimes the coincidence is explained by facts you're unaware of (e.g. three friends happen to mention a specific unusual topic during a week, and you don't know there was an article recently published on that topic, and all of them read it, or perhaps talked to someone who read it); (b) selective memory and reporting: a thousand things happen to you, most of which are common non-coincidences, and you remember only the freak coincidence.
If "synchronicity" is a fact, it's a fact about your imperfect mind—that you don't know the true causal connection and your memory is biased—not a fact about the world; I don't think there's some mysterious force deliberately throwing coincidences at you. But it seems that the guy who came up with the concept, Carl Jung, was (uncharitable explanation alert) so epistemically arrogant that he believed it was a mysterious fact about the world rather than about his imperfect mind.
Am I being uncharitable? Judge for yourself:
Synchronicity (German: Synchronizität) is a concept first introduced by analytical psychologist Carl G. Jung "to describe circumstances that appear meaningfully related yet lack a causal connection."[1] In contemporary research, synchronicity experiences refer to one's subjective experience that coincidences between events in one's mind and the outside world may be causally unrelated to each other yet have some other unknown connection.[2] [...]
Jung developed the theory of synchronicity as a hypothetical noncausal principle serving as the intersubjective or philosophically objective connection between these seemingly meaningful coincidences.[1][5] Mainstream science generally regards that any such hypothetical principle either does not exist or would not fall within the bounds of science.[6][7] [...]
Jung used the concept of synchronicity in arguing for the existence of the paranormal.[22] This idea was similarly explored by writer Arthur Koestler in his 1972 work The Roots of Coincidence[23] and was also taken up by the New Age movement.[6]
For this "enantiodromia", I would again say: It's a fact about the minds of those who make policies and were too unimaginative to think how they might go wrong, too incompetent to figure out how they would go wrong, or too careless to even try; not a fact about the world that there's some mysterious force that enjoys dramatic irony and tries to make your policies go wrong. Only some epistemically arrogant person would think it's a fact about the world... hey, guess what, it's Carl Jung again! I seriously did make the connection to "synchronicity" before I looked up the Wikipedia on enantiodromia. Behold:
Enantiodromia (Ancient Greek: ἐνάντιος, romanized: enantios – opposite and δρόμος, dromos – running course) is a principle introduced in the West by psychiatrist Carl Jung. In Psychological Types, Jung defines enantiodromia as "the emergence of the unconscious opposite in the course of time."[1] It is similar to the principle of equilibrium in the natural world, in that any extreme is opposed by the system in order to restore balance. When things get to their extreme, they turn into their opposite. Jung adds that "this characteristic phenomenon practically always occurs when an extreme, one-sided tendency dominates conscious life; in time an equally powerful counterposition is built up which first inhibits the conscious performance and subsequently breaks through the conscious control."[1]
However, in Jungian terms, a thing psychically transmogrifies into its shadow opposite, in the repression of psychic forces that are thereby cathected into something powerful and threatening.
Right. Well. I will just say that "people figuring out that a bounty on dead cobras incentivized them to breed cobras" does not require any invocation of unconscious opposites or psychic transmogrification to explain.
I see one advantage to talking about enantiodromia as a "principle": diplomacy. If people care about status, it may be easier to tell a would-be reformer "Have you considered that your proposal might hit this mysterious abstract principle?" than "Have you considered that you may be too naive, unimaginative, and/or incompetent to see how your proposal will go wrong?". (Note that I mean "incompetent" for the task, not compared to average; there may be policy failures that even the most competent person on Earth couldn't foresee.) Indeed, as I see these references to earlier discussion of the "principle" during history, I suspect some of them are cases where people expected to punished if they openly criticized policy, so they talked about an abstraction instead; I even suspect something similar might be upstream of this thread. (I doubt it would actually work as a tactic, to persuade those who would respond badly to regular criticism; but perhaps people tried it.)
But there are other ways to address the politeness aspect. You can say things like "For any important proposal, we should try to imagine how it could go wrong", mention perverse incentives if they seem relevant, mention the fun historical examples of screwups, and so on. I don't think I'm an expert at politeness, but I think it's a solvable problem. In any case, Less Wrong is not a place where people would talk about a supernatural force because it's easier to be polite that way.
I see one advantage to talking about enantiodromia as a "principle": diplomacy. [...] In any case, Less Wrong is not a place where people would talk about a supernatural force because it's easier to be polite that way.
LessWrong is a mix of a place where some people have nothing at stake and other people have a lot at stake.
There are conversation where important information could plausibly exchanged but those conversations don't happen because of various pressues. I do consider it useful to think how more of those conversations could happen. People are willing to share some information in private that they are not willing to publically share and narrowing that gap is useful.
Closely related to common libertarian/conservative objections to liberal (US sense) policies. I was actually taught this argument structure in law school - one interesting thing is you can lay out a whole series of policy arguments like this completely in the abstract, without reference to any specific controversy. Unfortunately, in any particular instance you can make this objection to a proposed policy, with superficial plausibility, whether or not the objection applies in that particular case.
There's no substitute for actually figuring out whether the objections apply in a particular case.
Imagine Chopper Meme:
[Senior] We should do some policy X which will help people with problem Y
[Junior] You'll hurt the people you're trying to help because of prospective market incentives
[Senior] You won't, because of market power
[Junior] You will, because public choice/the policy will be used for corruption
[Senior] You won't, because first-order effects dominate all these downstream effects including the public choice/corruption
etc etc
The name "enantiodromia" does not exactly lend itself to existing "in the public consciousness", does it? https://en.wikipedia.org/wiki/Streisand_effect is something people can easily relate to. "Perverse incentives" is another one. Both terms are used widely.
Both terms exist but I don't think they help me when discussing the merits of affirmative action policies.
I do agree that maybe enantiodromia is just too long of a word and to difficult to use and the principle would benefit from a more catchy name.
Hormesis is often the plausible mechanism. Or in Buddhist parlance, fighting demons makes them stronger.
Oxford Languages defines enantiodromia as “the tendency of things to change into their opposites, especially as a supposed governing principle of natural cycles and of psychological development”. While Plato’s works are so influential that the project of Western philosophy is sometimes described as footnotes to Plato, his discussion of enantiodromia is not part of our standard Western worldview. To the extent that the principle of enantiodromia exists, it matters individually, it matters for choosing the right public policy and it matters for AI safety.
One practical example of enantiodromia is a public policy with the goal of creating racial equality. In the 20th century, elite universities found that they admitted a higher proportion of Jewish people than people of other backgrounds. They were admitting more Jewish people because Jewish people scored on average higher than non-Jewish people. To cut down the number of Jewish admissions they thought up a clever way where they added subjective character judgments as an admission criterion which allowed them to cut down the percentage of Jewish admission.
In a world where the performance of a lawyer or doctor correlates with their SAT score if the SAT score is the only thing that matters for admissions, whether or not a lawyer or doctor is Jewish is not going to correlate strongly with their SAT scores. Once you have a policy, where Jewish people need a higher SAT score to be accepted, the Jewish lawyers and doctors will on average have a higher SAT score and thus higher competency.
This competency difference in turn reinforces stereotypes. In a world, where the admission criteria result in Jewish lawyers being on average more competent than non-Jewish lawyers the stereotype of Jewish lawyers being more competent is justified. A smart person who seeks a competent lawyer has an easier time determining whether the last name of the lawyer sounds Jewish than determining the SAT score of the lawyer.
In the end, the policy to create equality between Jewish people and non-Jewish created stereotypes about how Jewish people are unequal. This is enantiodromia.
Given that the dynamic of enantiodromia does not exist in the public consciousness it’s very hard to talk about enantiodromia plays out with modern social justice policies, so it’s left as an exercise for the reader to think through the details.
Celibacy as a means to reduce sexually driven misbehavior provided fertile ground for an environment of sexual abuse scandals.
Over in Buddism which has its own celibacy expectations for ordained monks we see an abundance of sex scandals as well even when they are of a different nature than the scandals of the catholic church.
Research driven by the desire for pandemic preparedness led to Chinese researchers doing gain-of-function experiments on Coronaviruses under biosafety level II (not suitable for protecting researchers against airborne viruses) in Wuhan. Shortly afterward we see an airborne Coronavirus emerging from Wuhan which a genome that looks at first glace inconsistent with expectations from evolutionary theory which causes a global pandemic.
In my native country of Germany, our population cares very deeply about climate change. While you might expect that this leads to lower CO2 output as other European countries like Great Britain or France where environmentalism is weaker, the opposite is the case. German environmentalism managed to shut down nuclear power plants which in turn led to higher CO2 emissions. While unsuccessful German environmentalists even fought the newly created Tesla plant near Berlin.
Enantiodromia clashes with enlightenment values. In the protest work ethic, hard work is supposed to pay off. Exerting effort is seen as a value in itself. In popular society, nobody wants to hear that efforts at Social Justice have enantiodromic effects. Investing effort into the program of Social Justice is seen as virtuous and discussions of the enantiodromic effects of those programs are seen as unvirtuous.
Eliezer’s efforts to reduce the discussion of the basilisk led to the concept spreading widely and now having a big section on the LessWrong Wikipedia page.
Closer home, in the EA community, we have no public discussion about enantiodromic effects either. In private conversation, I heard that the forces that Anna Salamon describes in her recent posts and comments on LessWrong, led to CFAR having some enantiodromic effects.
OpenAI which was founded partly to reduce AI X-risk might have increased risk by accelerating capability building.
A sense of treating numbers about problems very seriously makes it harder to talk about problems in numbers because there’s more at stake. This dynamic might have Marc Lipsitch when he talked about biosafety at EA global not talked about his numbers about the likelihood that lab accidents create a pandemic. Numbers that were comfortably shared in peer-reviewed articles were not comfortable to be shared on the stage of EA global.
Years ago, at one LessWrong meetup, we discussed how valuable it is to become a vegan. If I remember the numbers correctly, at the time Animal Charity Evaluators calculated that it costs roughly 43 Euro to offset not being vegan for a year. For anyone who felt that they were gaining more than 43 of value from not being a vegan, this provided a good argument for not being a vegan from an EA standpoint.
Animal Charity Evaluators got to a number that low because they wanted to convince people to donate, but in the process, they also signaled that the personal choice of becoming Vegan has a relatively low impact.
More generally, the commitment to EA as a brand makes it harder to talk about problems created by EA organizations. Self-identification with EA makes it harder to accept problematic aspects because they conflict with wanting to self-identify in a positive way.
Whenever we spend a lot of energy on interventions, we hope that those are effective and spend a lot of time discussing whether or not the intervention has the desired effect. Currently, we however don’t discuss the potential enantiodromic effects of our interventions. I hope that having a word for it, might make it easier to have discussions and hopefully easier to counteract the effects.