There might be a dark forest phenomenon with anti-totalitarianism activism.
I've seen a lot of people in EA saying things like "nobody is trying to prevent totalitarianism right now, therefore it's neglected" which leads them towards some pretty bonkers beliefs like "preventing totalitarianism is a great high-value way to contributing to making AGI end up aligned", because they see defending democracy as an endless crusade of being on the right side of history; and in reality, they have no clue that visibly precommitting to a potentially-losing side is something that competent people often avoid, or that the landscape of totalitarianism-prevention might already be pretty saturated by underground networks of rich and powerful people, who have already spent decades being really intense about quietly fighting totalitarianism, and aren't advertising the details of their underground network of allies and alliances.
In which case, getting EA or AI safety entangled in that web would actually mean a high probability of having everything you care about hijacked, appropriated, used as cannon fodder, or some other way of being steered off a cliff.
The part of the metaphor that feels most resonant to me here is "you're in a dark place and there's things you'd maybe expect to see, but don't, and the reason you don't see X is specifically because X doesn't want you to find it."
I think that instead of ending with "and the reason you don't see X is specifically because X doesn't want you to find it", it makes more sense to end with something more like "and the reason you don't see X is specifically because X doesn't want *many unknown randos with a wide variety of capabilities and tendencies* to find it".
Maybe the other people in the dark forest want to meet people like you! But there's all sorts of other people out there too, such as opportunistic lawyers and criminals and people much smarter or wealthier or aggressive than them.
... or an unglamorous place, like socially awkward corporate executives struggling at office politics (office politics aren't a good fit for them due to social awkwardness), but they keep at it because they're still too smart to fail at the office politics ...
That is a very interesting point. I never even considered before the possibility of someone just smart enough to keep afloat in office politics but not smart enough to transcend it. But when spelled out like that, it seems obvious that there must be a sizable cohort of middle managers and executives t... (read more)
There's a concept I first heard in relation to the Fermi Paradox, which I've ended up using a lot in other contexts.
Why do we see no aliens out there? A possible (though not necessarily correct) answer, is that the aliens might not want to reveal themselves for fear of being destroyed by larger, older, hostile civilizations. There might be friendly civilizations worth reaching out to, but the upside of finding friendlies is smaller than the downside of risking getting destroyed.
Even old, powerful civilizations aren't sure that they're the oldest and most powerful civilization, and eldest civilizations could be orders of magnitude more powerful still.
So, maybe everyone made an individually rational-seeming decision to hide.
A quote from the original sci-fi story I saw describing this:
(I consider this a spoiler for the story it's from, so please don't bring that up in the comments unless you use spoiler tags[1])
However this applies (or doesn't) to aliens, I've found it useful to have the "Dark Forest" frame in a number of contexts where people are looking at situation, and see something missing, and are confused. "Why is nobody doing X?", or "Why does X not exist?". The answer may be that it does exist, but is hidden from you (often on purpose).
I once talked to someone new to my local community saying "nobody is building good group houses that really help people thrive. I'm going to build one and invite people to it." I said "oh, people are totally building good group houses that help people thrive... but they are private institutions designed to help the people living there. The way they function is by creating a closed boundary where people can build high trust relationships."
A couple other example areas where Dark Forest Theorizing is relevant:
In some cases, a thing might not literally be hidden on purpose from you, but it's absence is still evidence of something systematically important. For example, "Why aren't the AI people making a giant mass movement to raise public awareness of AI?" Because many versions of a mass movement might turn out to be net-negative (i.e. causing political polarization which makes it harder to get the necessary bipartisan support to pass the relevant laws), and instead the people involved are focused on more narrow lobbying efforts.
(This is not to say that you can't make good public meetups, or successfully talk to AI companies and get them to change their ways, or that there aren't good ways to do mass public outreach on AI that are being neglected. But there may be some systemic difficulties you may be missing)
I do realize the cosmic existential threat aspect of the original metaphor is pretty overkill for some of these. The part of the metaphor that feels most resonant to me here is "you're in a dark place and there's things you'd maybe expect to see, but don't, and the reason you don't see X is specifically because X doesn't want you to find it."
I may have more to say on individual Dark Forests in the future. For now, I just want to present this as a model to keep in your back pocket and see where it's useful.
Spoiler tagged lines begin ">!"
See The Relationship Between the Village and the Mission