I like this post. What I really wish though was if I was better at explaining this to my friends and family. Has anyone on here ever had any success explaining this to an extent where you feel like the other person is really getting it? Perhaps I should become a truth cleric for a while and see if I can convert literal people on the street.
1Do not believe such things, for it is hurtful to others to believe such things!
Did you know that this is precisely the reasoning which, as Cynical Theories have shown, created stuff like cancel culture?
It’s usually much easier to bullshit value claims than epistemic claims.
Sure, if we compare the sets of all value claims with all epistemic claims. However, the controversial epistemic claims aren't typical, they're selected for both being difficult to verify and having obvious value implications. Consider the following "factual" claims that are hacking people's brains these days:
- that there are innate gender identities entirely independent of biological sex;
- that elite consensus undermines civilization by enforcing luxury beliefs;
- that so-called objective truthseeking perpetuates a system of oppression invented by dead white men;
- that safetyists are trying to stifle technological progress by means of totalitarian control.
It's not clear to me that "putting truth first" is a reliable enough defense for ordinary people in the face of that.
Except that it is a defense against the point #3. One might also do, say, a thought experiment with alien civilisations untouched by whites' hands and unaware about the oppression system.
The points #1, #2 and #4 can, in principle, be checked experimentally or thought-experimentally.
#2 implies that elite-generated information pieces propagate false beliefs which are status symbols and undermine civilisation. The original author of the term "luxury beliefs" also implied that the elites know that the beliefs are false and don't live according to said beliefs (e.g. claim that "monogamy is kind of outdated" while planning to become happily married). But the effects of raising kids alone or in a family can be learned by actually studying the two sets of kids.
#4 has the safetyists claim that humans are highly likely to create an AI who will take over the Earth and do terrible things like genocide or disempowerment. In order to refute said claim, one would have to explain why P(genocide or disempowerment|ASI is created)* value loss is smaller than the benefits[1] of having the ASI or that P(genocide or disempowerment|ASI is created) is negligible (think of estimates of P(LHC destroys the Earth or the universe)).
#1 requires that the gender identity expresses in something (e.g. by having a biological boy consistently prefer girly things, to play girly roles, etc.) IIRC proponents of the gender identity theory imply that conflict between the identity and social roles damages the psyche, and opponents claim that one has to rule out things like gender identity change being a suboptimal strategy against other unendorsed urges (see, e.g. Fiora Starlight's testimony) or outright a social contagion. One could, in theory, check the social contagion hypothesis by placing a new transfer transgender student into a collective of kids trained on different data sets.
What #1,#2,#4 have in common is that it is harder to check experimentally unless you are immersed with the area and the potential difficulty of publishing your results threatening to invalidate the dominant narrative. See also @TsviBT's discussion of graceful deference to experts and @Noosphere89's discuaaion of ways for people to turn ideologically crazy.
- ^
Said benefits could also be outweighed by Intelligence Curse-like risks.
One might also do, say, a thought experiment with alien civilisations untouched by whites’ hands and unaware about the oppression system.
Even though their supposed oppressor classes are unlikely to look like white males, that doesn't guarantee the absence of platonic toxic whiteness & masculinity.
What #1,#2,#4 have in common is that it is harder to check experimentally unless you are immersed with the area and the potential difficulty of publishing your results threatening to invalidate the dominant narrative.
Indeed.
@Noosphere89′s discuaaion of ways for people to turn ideologically crazy
As it happens, I had a bit of a back-and-forth with the author in the comments.
The other part is that partisans of the narrative overfocus on the other side's bad arguments because of most people not being able to check the arguments, and to be frank the entire area is a mess that I'm not willing to go down, and I instead focus on less-charged topics.
Like, all 4 of the examples are great demonstrations of why you need to be able to steelman your opponent, and one of the central problems in politics is people are trapped in a loop of destroying bad arguments instead of focusing on good arguments.
On a warm spring weekend, Jerry B wanders through Hyde Park. At a corner, he happens upon the Preacher Man, standing on a soapbox and proclaiming the Way of Truth.
Preacher Man: … and they will come to you and they will say “Do not believe such things, for it is hurtful to others to believe such things!”. Or they will say “Do not believe such things, for it is unpleasant to believe such things, and you can do nothing about it regardless!”. Or they will say “Do not believe such things, for it is hard to hide your beliefs, and others will punish you for it when they find out!”. And all these entreaties you must ignore; let them sway your belief neither in one direction nor the other, but rather set them aside and consider only the evidence of what is. Only by putting Truth before Utility will you be able to achieve the greatest utility available to you.
Jerry B: Now hold up, Preacher Man. I don’t think I buy what you’re selling. Let’s try an example.
Preacher Man: (Smiles approvingly.) You doubt my prescription, yet you neither brush it aside, nor blindly accept it; you argue back and make things concrete. That is the Way of Truth. Please, go on.
Jerry B: Consider belief in the afterlife. Sure, obviously there is no afterlife as a factual matter, but belief in the afterlife spares so much pain for so many people.
… and I will readily admit that, in practice, the utilitarian calculus is not obviously in favor of belief in the afterlife. Certainly various religions exploited that belief, and burned huge amounts of real resources in service of that belief. Likely the technologies available to actually fight back against death would have seen far greater investment far sooner without belief in the afterlife, and the world would be concretely better as a result.
… but let’s set aside the specifics of our world’s history, and focus on a thought experiment. Suppose a single person, sometime far back in history, could choose whether to believe in the afterlife. There’s no organized religion involved, trying to exploit her; nor will she spread her belief (which could cause an organized religion to spring into existence). And she doesn’t have significant options available to fight back against death anyway. The choice influences only her, and has little impact on her actions, just her own happiness. She will be happier if she believes in the afterlife. So the utilitarian calculus works out in favor of her believing in the afterlife.
Again, I don’t necessarily claim that belief in the afterlife works this way in reality. But the thought experiment shows that it’s at least possible for situations to come up in which truth must be set aside for maximum utility. And given that it’s possible, it would be rather surprising if such situations never came up in reality.
Preacher Man: We will get to my main answer shortly, but first I note that you’ve assumed a rather specific flavor of utilitarianism: hedonic utilitarianism. A non-hedonic utilitarian could declare “I want my lost loved ones to actually live on somewhere, not to merely believe they live on somewhere”; that would be a perfectly valid utility function. Indeed, from a decision-theoretic standpoint an agent which generally wants the world to be in preferred states, and not merely to believe the world is in preferred states, is considerably less dysfunctional; an agent which merely wishes to believe things will tend to wirehead.
Jerry B: Fair. And we could have a whole debate about the extent to which hedonic utilitarianism accurately describes human values. But is it cruxy?
You claimed earlier that “Only by putting Truth before Utility will you be able to achieve the greatest utility available to you.”. Does that claim rely on an assumption (or perhaps empirical observation) that human values are not well-modeled as hedonic utilitarianism? Or do you claim that even a hedonic utilitarian would not achieve the greatest utility available to them without putting Truth before Utility?
Preacher Man: I claim that even a hedonic utilitarian would not achieve the greatest utility available to them without putting Truth before Utility.
Jerry B: Does my thought experiment not directly disprove that claim? For the hypothetical hedonic utilitarian woman, deciding only for herself whether to believe in the afterlife, and with few or no realistically-available choices which depend on that belief… it sure seems like the utility calculus must go in favor of belief in the afterlife.
Preacher Man: I agree that the utility calculus for that hypothetical woman goes in favor of belief in the afterlife.
Jerry B: Yet you still claim that even a hedonic utilitarian would not achieve the greatest utility available to them without putting Truth before Utility?
Preacher Man: Indeed. Note that your thought experiment required assuming away much of how belief-in-the-afterlife works in the real world; that was load-bearing. I claim that in the real world, even a hedonic utilitarian would not achieve the greatest utility available to them without putting Truth before Utility. I do not make this claim about all hypothetical agents in all hypothetical worlds; I make this claim about humans in our world.
Jerry B: Quite the claim. How will you defend it?
Preacher Man: (Laughs.) I am no lawyer; my goal is not to defend a claim. Rather, my goal is to communicate the model, so you can see that it is at least a coherent way-the-world-could-be, a hypothesis to which some probability is assigned. And then you will go forth and judge the truth of the claim for yourself.
Jerry B: Very well. What strange shape might our particular world take, such that even a hedonic utilitarian would not achieve the greatest utility available to them without putting Truth before Utility?
Preacher Man: Let’s briefly use an analogy: what happens when you connect a computer running Windows XP directly to the internet and turn off the firewall, in 2024?
Jerry B: Ten minutes later, it will be infected by malware, and within something like hours to days it will cease to meaningfully be under your control at all. It will instead be under the control of a botnet, or some spyware waiting for you to open a bank website, or whoever won the battle for control of the computer.
Preacher Man: Exactly. Heck, even with a firewall on a Windows XP machine is a prime target.
Now for the analogy: human brains are like Windows XP machines. Most of our brains are exposed to the internet, and to human culture more generally. We’re prime targets for memetic “viruses”. And a Truth-first commitment is like a firewall.
Jerry B: Ok, I’m starting to see the analogy here. But why a Truth-first commitment specifically?
Preacher Man: As the saying goes: “First they came for the epistemology. We don’t know what happened after that.”. Getting a little more gearsy about it…
Insofar as a meme is centered on a factual claim, the claim gets entangled with lots of other facts about the world; it's the phenomenon of Entangled Truths, Contagious Lies. So unless the meme tries to knock out a person's entire epistemic foundation, there's a strong feedback signal pushing against it if it makes a false factual claim.
But the Entangled Truths phenomenon is epistemic; it does not apply nearly so strongly to values. If a meme claims that, say, it is especially virtuous to eat yellow cherries from Switzerland... well, that claim is not so easily falsified by a web of connected truths.
As a result, parasitic memeplexes tend to be mostly claims about values (i.e. what’s good, healthy, cringe, etc) as opposed to epistemic claims, because value claims are much easier to bullshit.
Jerry B: Pause a moment, let me check that your abstract description actually binds to the example of belief in the afterlife.
As a purely factual matter, the simplest versions of an afterlife would have all sorts of observable effects in our world, which we do not observe. “Entangled truths”, as you say. One can go through some fairly precise mental gymnastics to hypothesize precisely the sort of afterlife which is not observable at all, but then that itself is rather suspicious.
That said, religious memeplexes do make factual claims about the afterlife! And other factual claims too, though admittedly the factual claims seem to be steadily rolled back century after century, while the value claims seem to have a much longer shelf life (though they too evolve with the memetic tides). So I suppose you’ll claim the process of religions’ factual claims being falsified over time is exactly the sort of thing you’re talking about?
Preacher Man: That is what I’ve been talking about so far, but there’s a big loophole.
It’s usually much easier to bullshit value claims than epistemic claims. But that means there’s a lot of memetic selection pressure looking for ways to bullshit an epistemic claim by replacing it with a value claim. In particular, there’s lots of memetic selection pressure for arguments that it’s virtuous, or beneficial, or good, or [etc] to believe X, for various value claims X. Because any time a meme could reproduce more efficiently by spreading a false epistemic claim, replacing that epistemic claim with a value claim is a likely-easier alternative fitness strategy.
In the case of the afterlife, that would mean that there’s lots of memetic selection pressure for arguments that it’s virtuous/beneficial/good/etc to believe in an afterlife. That sort of argument can sidestep the “epistemic firewall”, allowing a memeplex to exploit factual claims as part of its reproduction cycle, by motivating the host to not question those factual claims (because the host believes that e.g. it’s beneficial to believe in the afterlife, so they don’t want to question it).
Jerry B: Ok, but it might in fact be beneficial to believe in the afterlife.
Preacher Man: Maybe. But you know what else is beneficial? Having a firewall in place, which blocks out any attempts to update your epistemic beliefs based on what’s virtuous/beneficial/good/[etc] to believe. I.e. “truth first”.
Today’s world is an absolute hotbed of memeplexes, evolving far more quickly than our own brains. We are the Windows XP computers in this environment. In practice, if you don’t put truth first, if you don’t strictly firewall attempts to update your factual beliefs based on arguments about what’s good to believe (as opposed to what’s true), you’re going to get hacked, and it’s not going to take very long. And once hacked like that, you’re not going to be able to fix it easily, because value claims have much weaker feedback from reality than factual claims.
I wish to believe that (it's beneficial to believe in the afterlife) if-and-only-if (it is beneficial to believe in the afterlife). But if that belief is wrong, it's going to be a hard one to get feedback on, and I know it's under attack by the memeplexes. So even if I did think it's probably beneficial to believe in the afterlife, I wouldn't just go believing in the afterlife. At a policy level, it's probably worth it to keep the epistemic firewall in place.