Epistemic Status: pretty confident

Zvi recently came out with a post “You Have the Right to Think”, in response to Robin Hanson’s “Why be Contrarian?”, itself a response to Eliezer Yudkowsky’s new book Inadequate Equilibria. All of these revolve around the question of when you should think you “know better” or can “do better” than the status quo of human knowledge or accomplishment.  But I think there’s a lot of conflation of different kinds of “should” going on.

Yudkowsky’s book, and Hanson’s post, are mostly about epistemic questions — when are you likely to get the right answer by examining an issue yourself vs. trusting experts?

Inadequate Equilibria starts with the canonical example of when you can’t outperform the experts — betting on the stock market — and explains about efficient markets, and then goes on to look into what kinds of situations deviate from efficient markets such that an individual could outperform the collective intelligence of everyone who’s tried so far.  For instance, you might well be able to find a DIY treatment for your health problem that works better than anything your doctor would prescribe you, in certain situations — but due to the same incentive problems that prevented medical consensus from finding that treatment, you probably wouldn’t be able to get it to become the standard of care in the mass market.

Hanson mostly agrees with Yudkowsky’s analysis, except on some points where he thinks the argument for individual judgment being reliable is weaker.

Zvi seems to be talking about a different thing altogether when he talks about the “rights” that people have.

When he says “You have the right to disagree even when others would not, given your facts and reasoning, update their beliefs in your direction” or “You have the right to believe that someone else has superior meta-rationality and all your facts and reasoning, and still disagree with them”, I assume he’s not saying that you’d be more likely to get the right answer to a question in such cases — I think that would be false.  If we posit someone who knows better than me in every relevant way, I’d definitionally be more likely to get the right answer by listening to her than disagreeing with her!

So, what does it mean to have a right to disagree even when it makes you more likely to be wrong?  How can you have a right to be wrong?

I can think of two simple meanings and one subtle meaning.

The Right To Your Opinion

The first sense in which you “have a right to be wrong” is social and psychological.

It’s a basic tenet of free and pluralistic societies that you have the legal right to believe a false thing, and express your belief.  It is not a crime to write a horoscope column.  You can’t be punished by force just for being wrong.  “Bad argument gets counterargument.  Does not get bullet.  Never.  Never ever never for ever.”

And tolerant, pluralist cultures generally don’t believe in doing too much social punishment of people for being wrong, either.  It’s human to make mistakes; it’s normal for people to disagree and not be able to resolve the disagreement; if you shame people as though being wrong is horribly taboo, your community is going to be a more disagreeable and stressful place. (Though some communities are willing to make that tradeoff in exchange for higher standards of common knowledge.)

If you are regularly stressed out and scared that you’ll be punished by other people if they find out you believe a wrong thing, then either you’re overly timid or you’re living in an oppressive environment.  If fear of punishment or ostracism comes up regularly when you’re in the process of forming an opinion, I think that’s too much fear for critical thinking to work properly at all; and the mantra “I have the right to my opinion” is a good counterweight to that.

Discovery Requires Risking Mistakes

The second sense in which you have a “right to be wrong” is prudential.

You could ensure that you’d never be wrong by never venturing an opinion on anything.  But going all the way to this extreme is, of course, absurd — you’d never be able to make a decision in your life!  The most effective way to accomplish any goal always involves some decision-making under uncertainty.

And attempting more difficult goals involves more risk of failure. Scientists make a lot of hypotheses that get falsified; entrepreneurs and engineers try a lot of ideas that don’t work; artists make a lot of sketches that wind up in the wastebasket.  Comfort with repeated (hopefully low-stakes) failure is essential for succeeding at original work.

Even from a purely epistemic perspective, if you want to have the most accurate possible model of some part of the world, the best strategy is going to involve probabilistically believing some wrong things; you get information by testing guesses and seeing where you’re mistaken.  Minimizing error requires finding out where your errors are.

Note, though, that from this prudential perspective, it’s not a good idea to have habits or strategies that systematically bias you towards being wrong.  In the “right to your opinion” sense, you have a “right” to epistemic vices, in that nobody should be attacking you for them; but in this goal-oriented sense, they’re not going to help you succeed.

Space Mom Accepts All Her Children

The third sense in which you have a “right to be wrong” is a little weirder, so please bear with me.

There’s a mental motion you can do, when you’re trying to get the right answer or do the right thing, where you’re trying very hard to stay on the straight path, and any time you slip off, you violently jerk yourself back on track.  You have an aversion to wrongness.

I have an intuition that this is…inefficient, or mistaken, somehow.

Instead, there’s a mental motion where you have peripheral vision, and you see all the branching paths, and consider where they might go — all of them are possible, all of them are in some cosmic sense “okay” — and you perform some kind of optimization procedure among the paths and then go along the right path smoothly and without any jerks.

Or, consider the space of all mental objects, all possible thoughts or propositions or emotions or phenomena or concepts.  Some of these are true statements; some of them are false statements. Most of them are unknown, or not truth-apt in the first place.  Now, you don’t really want to label the false ones as true — that would just be error.  But all of them, true or false or neither or unknown, are here, hanging like constellations in this hypothetical heaven. You can look at them, consider them, call some of them pretty.  You don’t need to have an aversion response to them. They are “valid”, as the kids say; even if they don’t have the label “true” on them, they’re still here in possibility-space and that’s “okay”.

In a lot of traditions, the physical metaphor for “good” is high and bright.  Like the sun, or a mountaintop. The Biblical God is described as high and bright, as are the Greek Olympians or the Norse gods; in Indian and Chinese traditions a lot of divine or idealized entities are represented as high and bright; in ordinary English we talk about an idealistic person as “high-minded” and everybody knows that the “light side of the Force” is the side of the good guys.

To me, the “high and bright” ideal feels connected to the pattern of seeking a goal, seeking truth, trying not to err.

But there are also traditions in which “high and bright” needs to be balanced with another principle whose physical metaphor is dark and vast.  Like the void of space, or the deeps of the sea.  Like yin as a complement to yang, or prakrti as a complement to purusa, or emptiness as a complement to form.  

The “high and bright” stuff is value — knowledge, happiness, righteousness, the things that people seek and benefit from.  The “dark and vast” stuff is possibility.  Room to breathe. Freedom. Potential. Mystery. Space.

You can feel trapped by only seeking value — you can feel like you lack the “space to be wrong”.  But it’s not really that you want to be wrong, or that you want the opposite of value; what you want is this sense of “enough room to move”.

It’s something like Keats’ “negative capability“:

…at once it struck me, what quality went to form a Man of Achievement especially in Literature & which Shakespeare possessed so enormously—I mean Negative Capability, that is when man is capable of being in uncertainties, Mysteries, doubts, without any irritable reaching after fact & reason—Coleridge, for instance, would let go by a fine isolated verisimilitude caught from the Penetralium of mystery, from being incapable of remaining content with half knowledge.

or something like the “mother” Ahab perceives behind God:

But thou art but my fiery father; my sweet mother, I know not. Oh, cruel! what hast thou done with her? There lies my puzzle; but thine is greater. Thou knowest not how came ye, hence callest thyself unbegotten; certainly knowest not thy beginning, hence callest thyself unbegun. I know that of me, which thou knowest not of thyself, oh, thou omnipotent. There is some unsuffusing thing beyond thee, thou clear spirit, to whom all thy eternity is but time, all thy creativeness mechanical. Through thee, thy flaming self, my scorched eyes do dimly see it. Oh, thou foundling fire, thou hermit immemorial, thou too hast thy incommunicable riddle, thy unparticipated grief.

The womb of nature, the dark vastness of possibility — Space Mom, so to speak — is not the opposite of reason and righteousness so much as the dual to these things, the space in which they operate.  The opposite of being right is being wrong, and nobody really wants that per se.  The complement to being right is something like “letting possibilities arise” or “being curious.” Generation, as opposed to selection.  Opening up, as opposed to narrowing down.

The third sense in which you have the “right to be wrong” is a lived experience, a way of thinking, something whose slogan would be something like “Possibility Is.”

If you have a problem with “gripping too tight” on goals or getting the right answer, if it’s starting to get oppressive and rigid, if you can’t be creative or even perceive that much of the world around you, you need Space Mom.  The impulse to assert “I have the right to disagree even with people who know better than me” seems like it might be a sign that you’re suffocating from a lack of Space Mom.  You need openness and optionality and the awareness that you could do anything within your powers, even the imprudent or taboo things.  You need to be free as well as to be right.

New Comment
10 comments, sorted by Click to highlight new comments since:

The "Space Mom" part seems like what exploration/exploitation meta-algorithm feels like from the inside. To do quality exploration you need to shut down the inner voice of exploitation.

Also relevant: https://www.physics.ohio-state.edu/~kilcup/262/feynman.html

I'm kind of a sucker for polytheistic playfulness, but even so I love your literary grounding and evocative definition of Space Mom, as an initially scary but ultimately calming guide through her chaotic domain.

If I want to reach back to earlier versions, there's also Eris (disliked by the Greeks but beloved of Discordians), and arguably even better than Eris is the Sumerian goddess Tiamat!

Tiamat is the "the chaos of primordial creation", who is female but with strong reptilian themes as well. In Sumerian mythology Tiamat is killed by Marduk, the king of the gods whose crown is made out of a ring of eyes that vigilantly look in every direction, and whose power is to speak the truth.

After killing her, Marduk used her body to make the entire world. If you contrast "Tiamat vs Space Mom" preferring the Space Mom iconography would argue that you need to make friends with chaos. You need to work on desensitizing yourself to fear so you can freely look into the shadows, not clench up and get ready to fight.

Moden visual adaptations of Tiamat lean on outer space motifs but her original context for the Sumerians was salt water, and the ocean, and especially the estuary where salt water and fresh water chaotically mix on a cycle a bit less than 13 hours long that is somehow related to the moon.

In MtG iconography, Tiamat is Blue/Black and Marduk might be White/Black, with the separation along the White/Black axis representing simplistic dualisms... the emotional triumph of understanding over fear, moral triumph of good over evil, the familial triumph of males over females, the astronomical importance of the sun over the moon, and the political triumph of law over crime.

Isbell's snake detection hypothesis argues that primate visual acuity evolved over the last ~100M years (with detection of literal predatory snakes as the biggest driver of the pixel count in our eyeballs) thereby installing "snake monsters" as a symbol that our brains reliably imagine and pareidolically react to because false negatives on the snake monster detection task are very expensive. Some of the "symbolicly loose" psychology people (who don't write off Jung as "not even wrong") have picked up on this and used it to justify connecting snake iconography to mythology and therapeutic stories.

Tiamat is THE dragon in Sumerian mythology. In Chinese mythology, dragons are often good, but fickle. In Western mythology dragons used to be mostly bad, but in more recent cinema they become the beloved black pets that let social misfits fly above the ocean who might grow up to talk like Sean Connery as they help con men gain a conscience and topple unjust kings.

Space Mom (seen as a friendlier version of Tiamat) is a more balanced take on blue far mode. She is never "here and now". She is always "out around the edges", either requiring a journey to reach, or else part of ancient history and possibly the deep future. She represents the dramatic schism between future eutopias and dystopias. And back when you acquired your mother tongue, in a time you forgot, back when you weren't even you (as far as continuity of memory goes) back when your only words were <crying> and <not crying>, she was with you and helped you to learn... and if you calm your fear of the dark she still can.

This helped me see how Space Mom as described might be primarily something other than terrifying. Usually descriptions of it read to me like fully general praise of hiding things, which is pretty deeply sketchy. But, exploring places that aren't well-lit yet makes sense.

A Buddhist friend of mine says that Space Mom is a good match for Prajnaparamita.

I'd never heard of her before, but maybe something resonates there.

Cool link! I had not heard of her before but I see the echoes. To summarize some of the resonances I think I see...

I noticed that the Sutra about her is the Heart Sutra, and it arose as part of the Mahayana correction to the early ascetic "small raft" Buddhism, and was claimed to have been the secret teachings of Buddha that couldn't be taught in the initial version of Buddhism because the people were not ready...

It is claimed to have been technically there at the beginning, but not in an obvious way.

The secret teachings were mythologically kept by the king of the snakes in his underwater kingdom for a full turn of history, until a reincarnation of Buddha arrived named Nagarjuna, where "Naga" means snake and "Arjuna" means something like "bright shining silver" and is the name of the central hero of the Bhagavad Gita. Thus Nagarjuna, the teacher of the lesson, had a name that basically meant "Illuminated Snake Hero".

The ideas were mythologically acquired by: going underwater, making friends with the snake king, then studying the snake king's secrets (that he got from Buddha).

These lessons, that Prajnaparamita is the embodiment of, are given the concept handle of "shunyata" ("emptiness") and basically seems to be a denial of local naive realism? That is to say: there are no permanent things whose meaning and reality are independent of context. So if you take this seriously and ask "But what's the context?" over and over for anything and everything, recursively, then perhaps eventually you always get to Prajnaparamita as the contextual "Mother of All".

Epistemically speaking, chasing Prajnaparamita is valuable, because you learn the context of your current naively local truth. However you'll never get to her and go past her, because she represents the edge of knowledge... she is always "the farther away context of which you are currently ignorant". As you learn, she always retreats into the background, representing the new edge of knowledge.

Prajnaparamita's name literally means "perfect wisdom", and while she is technically unattainable, it is useful to try to approach her :-)

If you look at the emotional differences in the symbolic choice of Tiamat vs Prajnaparamita, then Tiamat pushes all the ideas into a single fundamentally bad kind of watery chaos that must be destroyed in a violent way for goodness and masculine knowledge to triumph. On the other hand Prajnaparamita has all the emotionally negative aspects sublimated into the process of pursuing her (into the watery domain of the snake king), and is seen as fundamentally good in herself.

Both kinds of symbolism are "mixed", but one valorizes the heroic killing and re-use of "scary female mysteries" while the other justifies "painful exploration" as worthwhile pursuit of the ultimate ineffable female context.

Calling out some of these echoes, I think I see different arrangements of many of the same concepts. Also, the arrangement of the concepts in the "Space Mom" framing seems closer to Prajnaparamita than Tiamat.

If you generate predictions by looking up market values, that means three things:

  1. You're never going to make any money on the market, not even a tiny bit
  2. You're never going to contribute to the market
  3. You're never really going to learn what the market knows about underlying causes

Because of (2), if everyone did this, there would be no market.

You need your own machine for generating predictions. Here, I'm borrowing a metaphor from Ronny Fernandez, who says that he keeps track of predictions because he needs to debug the program that generates them.

You don't need to believe the resulting predictions. You can make predictions like, "I think that the probability of a Trump re-election is .99 times the probability assigned by this prediction market, plus .01 times the probability output by my machine." You basically believe the market's prediction, but have some small way of contributing.

I think (3) is more important though. If the market is consistently right about something, there exists a machine that outputs good predictions. That machine may even be a causal model that gives you some deep insight about the world. And you're never going to have it in your head if all you do is make market-based opinions.

So I think that's another sense of "you have a right to be wrong." You have a right to keep tinkering with your own prediction-machine in your garage, even if it's not yet competitive with the finished products on the market.

One way to employ Space Mom might be with how confidently you believe expert concensus, in particular given that experts rarely give their confidence levels. For instance:

A. Expert concensus says that horoscopes are bunk. I believe it! I have a tight confidence interval on that.

B. Expert concensus says that hospitals provide significant value. I believe that too! But thanks to Robin Hanson, I'm less confident in it. Maybe we're mostly wasting our healthcare dollars? Probably not, but I'll keep that door in my mind open.

----

Separately, I think the frustrating thing about Hanson's piece was that he seemed to be making an isolated demand for rigor in that Eliezer prove in an absolute sense that he can know he is more rational than average before he gets his "disagreement license."

"You could be deceiving yourself about having valid evidence or the ability to rationally consider it" is a fully general argument against anything, and that's what it felt like Hanson was using. In particular because Eliezer specificially mentioned testing his calibration against the real world on a regular basis to test those assumptions.

This is a nice post. I imagined a little shining marble hovering above an infinite obsidian plane when you were describing the yin/yang dichotomy, and I think that saying these things are not opposites but duals is a brilliant choice of words. I'm going to keep this dichotomy in mind as a lens through which to see various situations.

Cute little side note: keeping "lenses through which to see various situations" in your toolbox is a Space Mom move. It's a thing you do that helps you generate possibilities or ideate, rather than a pair of scissors that helps you prune. I think I used to not see why a "lens" would ever be useful, and this post helped me understand it.

Two additional senses in which a "right to be wrong" might be justified: in differing risk preferences, individually, or the usefulness of holdout populations, societally.

[+][comment deleted]00