Somewhere in Polybius is a line about character. Men have characters. Yours may lead you to greatness, but when times change and your character is ill-suited to new circumstances, you remain the same. Disaster follows.

I agree that there is a problem. I think that it has been recognised long ago and has no solution.

I was very struck that in our imaginations we are always the good guys. I have developed serious doubts about this. Looking back on my life it struck me that as a young man I was rather attracted by authoritarian governments. If I had been a 13 year old German boy when Hitler came to power in 1933 I would have worshipped the man. He would have been my hero and I would have believed all that stuff about Jews.

I'm not trying to shock or abase myself. There is actually an element of smugness as I feel ahead of head of the pack in realising that the good guys and the bad guys are mostly just ordinary guys and land on one side or the other because it is hard to transcend ones circumstances. We know that Hitler was popular in 1933. What must also be true is that if we had been young and German we might well have fallen under his spell. We might perhaps have been proud of our philosophical sophistication and knowledge of Heidegger but it wouldn't have helped.

I see the post as aiming way too high. If you are crap at lying that has consequences, perhaps good, perhaps bad. Time will tell, don't worry about it. The more interesting question is about how to transcend ones circumstances. How does one avoid being one of the brownshirts?

When we worry here about promoting maximally accurate beliefs it is self-deception that is the principle concern, not the deception of others. Eliezer posted about affective death spirals. An example could be being so impressed by the Fuehrer that we believe everything he says. Later, we notice that he says an awful lot. We notice, wtih some surprise, that It is all true. What a genius! And round we go. I've understood Eliezer to be addressing the issue of how we can tell from the inside; how we can escape our own affective death spirals. That is a very different problem from seeing the problem in others and opposing them.

'Somewhere in Polybius...'

See, that's where your post first goes wrong. This is LW: you need to quote either anime, SF, fanfiction, or something preferably all three. I'll spot you a suitable quote, but next time you're on your own!

'Somewhere in Dune Messiah, Frank Herbert writes "A creature who has spent his life creating one particular representation of his selfdom will die rather than become the antithesis of that representation."...'

Degrees of Radical Honesty

by MBlume 3 min read31st Mar 200951 comments


The Black Belt Bayesian writes:

Promoting less than maximally accurate beliefs is an act of sabotage. Don’t do it to anyone unless you’d also slash their tires, because they’re Nazis or whatever.

Eliezer adds:

If you'll lie when the fate of the world is at stake, and others can guess that fact about you, then, at the moment when the fate of the world is at stake, that's the moment when your words become the whistling of the wind.

These are both radically high standards of honesty. Thus, it is easy to miss the fact that they are radically different standards of honesty. Let us look at a boundary case.

Thomblake puts the matter vividly:

Suppose that Anne Frank is hiding in the attic, and the Nazis come asking if she's there. Harry doesn't want to tell them, but Stan insists he mustn't deceive the Nazis, regardless of his commitment to save Anne's life.

So, let us say that you are living in Nazi Germany, during WWII, and you have a Jewish family hiding upstairs. There's a couple of brownshirts with rifles knocking on your door. What do you do?

I see four obvious responses to this problem (though there may be more)

  1. "Yes, there are Jews living upstairs, third door on the left" -- you have promoted maximally accurate beliefs in the Nazi soldiers. Outcome: The family you are sheltering will die horribly.
  2. "I cannot tell you the answer to that question" -- you have not deceived the Nazis. They spend a few minutes searching the house. Outcome: The family you are sheltering will die horribly.
  3. "No, there are no Jews here" -- your words are like unto the whistling of the wind. The Nazis expect individuals without Jews in their homes to utter these words with near certainty. They expect individuals with Jews in their homes to utter these words with near certainty. These words make no change in P(there are Jews here) as measured by the Nazis. Even a couple of teenaged brownshirts will possess this much rationality. Outcome: The family you are sheltering will die horribly.
  4. Practice the Dark Arts. Heil Hitler enthusiastically, and embrace the soldiers warmly. Thank them for the work they are doing in defending your fatherland from the Jewish menace. Bring them into your home, and have your wife bring them strong beer, and her best sausages. Over dinner, tell every filthy joke you know about rolling pennies through ghettos. Talk about the Jewish-owned shop that used to be down the street, and how you refused to go there, but walked three miles to patronize a German establishment. Tell of the Jewish moneylender who ruined your cousin. Sing patriotic songs while your beautiful adolescent daughter plays the piano. Finally, tell the soldiers that your daughter's room is upstairs, that she is shy, and bashful, and would be disturbed by two strange young men looking through her things. Appeal to their sense of chivalry. Make them feel that respecting your daughter's privacy is the German thing to do -- is what the Feurer himself would want them to do.  Before they have time to process this, clasp their hands warmly, thank them for their company, and politely but firmly show them out.  Outcome: far from certain, but there is a significant chance that the family you are sheltering live long, happy lives.

I am certain that YVain could have a field day with the myriad ways in which response 4 does not represent rational discourse. Nonetheless, in this limited problem, it wins.

(It should also be noted that response 4 came to me in about 15 minutes of thinking about the problem. If I actually had Jews in my attic, and lived in Nazi Germany, I might have thought of something better).


What if you live in the impossible possible world in which a nuclear blast could ignite the atmosphere of the entire earth? What if you are yourself a nuclear scientist, and have proven this to yourself beyond any doubt, but cannot convey the whole of the argument to a layman? The fate of the whole world could depend on your superiors believing you to be the sort of man who will not tell a lie.  And, of course, in order to be the sort of man who would not tell a lie, you must not tell lies.

Do we have wiggle room here? Neither your superior officer, nor the two teenaged brownshirts, are Omega, but your superior bears a far greater resemblance. The brownshirts are young, are ruled by hormones. It is easy to practice the Dark Arts against them, and get away with it. Is it possible to grab the low-hanging fruit to be had by deceiving fools (at least, those who are evil and whose tires you would willingly slash), while retaining the benefits of being believed by the wise?

I am honestly unsure, and so I put the question to you all.

ETA: I have of course forgotten about the unrealistically optimistic option:

5: Really, truly, promote maximally accurate beliefs. Teach the soldiers rationality from the ground up. Explain to them about affective death spirals, and make them see that they are involved in one.  Help them to understand that their own morality assigns value to the lives hidden upstairs.  Convince them to stop being nazis, and to help you protect your charges.

If you can pull this off without winding up in a concentration camp yourself (along with the family you've been sheltering) you are a vastly better rationalist than I, or (I suspect) anyone else on this forum.