Another monthly installment of the rationality quotes thread. The usual rules apply:

  • Please post all quotes separately, so that they can be upvoted or downvoted separately. (If they are strongly related, reply to your own comments. If strongly ordered, then go ahead and post them together.)
  • Do not quote yourself.
  • Do not quote comments or posts from Less Wrong itself or from Overcoming Bias.
  • No more than 5 quotes per person per monthly thread, please.
Rationality Quotes February 2013
New Comment
566 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

"If all your friends jumped off a bridge, would you jump too?"

"Oh jeez. Probably."

"What!? Why!?"

"Because all my friends did. Think about it -- which scenario is more likely: every single person I know, many of them levelheaded and afraid of heights, abruptly went crazy at exactly the same time... ...or the bridge is on fire?"

Randall Munroe, on updating on other people's beliefs.

[-]satt150

Dilbert dunnit first!

(Seeing that strip again reminds me of an explanation for why teenagers in the US tend to take more risks than adults. It's not because the teenagers irrationally underestimate risks but because they see bigger benefits to taking risks.)

Let me just put the text string ‘xkcd’ in here, because I was going to add this if nobody else had, and it's lucky that I found it first.

Oh, and there's more text in the comic than what's quoted, and it's good too, so read the comic everybody!

1A1987dM
See also this Will_Newsome comment. (I incorrectly remembered that it said something like “If all your friends jumped off a bridge, would you jump too?” “If all of them survived, I probably would.”)
-1olibain
The " every single person I know, many of them levelheaded and afraid of heights, abruptly went crazy at exactly the same time" scenario should be given some credence in human society; there is such a thing as puberty. The definition of puberty being " every single person I know abruptly went crazy at exactly the same time, including me".

It’s nice to elect the right people, but that’s not the way you solve things. The way you solve things is by making it politically profitable for the wrong people to do the right things.

-- Milton Friedman

No one can be good for long if goodness is not in demand.

-- Bertold Brecht

(I'm always amused when people of opposite political views express similar thoughts on society.)

Also:

The aim of science is not to open the door to infinite wisdom, but to set some limit on infinite error.

0Eugine_Nier
I think the Brecht quote is somewhat misleading. The problem is not that not enough people want/demand goodness, the problem is that it is too easy to profit by cheating without getting caught.
4AlexSchell
This solution only works if you are in the special position of being able to make institutional design changes that can't be undone by potential future enemies. Otherwise, whose "right things" will happen depends on who is currently in charge of institutional design (think gerrymandering).
1Sengachi
Then try to make it politically profitable to help sustain those changes you make. Make it so painfully obvious that the only reason to remove those changes would be for one's unethical gain that no politician would ever do so. The problem then though, is that people end up just not caring enough.
5AlexSchell
What you're describing is exactly the position of being able to make institutional design changes that can't be undone by potential future enemies. This position is "special" not only because the task is very difficult, but also because you have to be the first to think of it.
1Estarlio
Couldn't I also set up the system to try to exclude the wrong people from ever getting power? It seems to me that computers get better at detecting liars, and we have an ease of fact checking on things now we never used to have, and conflicts of interest are generally relatively easily seen, and we've got all this research about how influence functions... In short that we've made a lot more progress on the judging people front, than we have on the side of designing procedures and regulations that suit us and also serve as one-way functions.
8fubarobfusco
Not if having power over others turns the right people into the wrong people.
5Richard_Kennaway
No. No-one can set up the system. The most that anyone can do is introduce a new piece into the game, pieces like Google, or Wikipedia, or Wikileaks.
-4ChristianKl
That mentality is probably why US politics is as corrupt as it is at the moment. Electing people who aren't corrupt to replace corrupt people is very valuable if your goal is to have a well governed country. If you have the political goals of Milton Friedman it might not be. If you want politicians to be corportate friendly than you make it politically profitable for them to do so by making it easy for companies to bribe them.

I think the spirit of the quote is that instead of counting on anyone to be a both benevolent and effective ruler, or counting on voters to recognize such things, design the political environment so that that will happen naturally, even when an office is occupied by a corrupt or ineffective person.

0woodside
This idea is primarily why I'm skeptical of the effectiveness of institutions like the federal reserve (despite not being a subject matter expert). It seems pretty clear that in order to be effective the leadership has to be comprised of people that are not only exceptionally brilliant, but exceptionally benevolent as well.
-6ChristianKl
8Nornagest
The nice thing about working with incentives is that they're pretty stable relative to political leanings. I'd expect a given person's perceptions of politicians' level of corruption or incompetence or any other negative adjective you can think of to depend almost entirely on party affiliation, but you can actually leverage that to get changes in incentive structures passed: just frame it as necessary to curb the excesses of those guys over there, you know, the ones you hate. And in any case the quote works just as well for the governed. As anyone who's ever moderated a large forum can tell you, playing with incentives works almost embarrassingly well and quickly compared to working on sympathy or respect for authority. Of course, it's also harder to do.
2HalMorris
That sounds very intriguing. Can you give some example of how you've used "playing with incentives" successfully to (I assume - correct me if I'm wrong) maintain a productive forum? That might be very enlightening - seriously, no irony here.

Simplest positive example I can think of offhand: if there's lots of content-free posting going on and you want it to go away, changing the board parameters so that user titles are no longer based on postcount goes a surprisingly long way.

Simplest negative example I can think of: if you think there's too much complaining going on (I didn't, but the board owner at the time did), allocating a subforum for complaints will only make things worse. Even if you call it something like "Constructive Criticism".

1HalMorris
Sorry, I've never run a forum. Is there any easy place to learn enough to make "user titles are no longer based on postcount" make sense to me (unless you want to take the time to explain it). I really am very interested.
5Nornagest
Sure. One feature in phpBB and several other popular bulletin board packages (but not in reddit or Slashdot or any of their descendants) is the ability to set user titles: little snippets of descriptive text that get displayed after a user's handle and which are usually intended to give some information about their status in the forum. The most common arrangement is to have a couple of special titles for administrative positions (say, "mod" and "admin"), then several others for normal users that're tiered based on the number of posts the user's written, i.e. postcount: a user might start with the title "newbie" or "lurker", then progress through five or six cutely themed titles as they post more stuff. It's common for admins to change the exact titles and the progression pattern to suit the needs of the forum (a roleplaying forum for example might name them after monsters of increasing power), but uncommon to change the basic scheme. You may notice that this doesn't differentiate on post quality.
4Baruta07
Look up some of the karma discussions on this very site.
-7ChristianKl
4Viliam_Bur
If you have corrupt politicians, blame the voters. The politicians did not vote themselves into the office. (Unless they own the vote-counting machines factory.) I guess the quote suggests that "making it politically profitable for the wrong people to do the right things", whatever precisely that means, could still be easier than replacing the whole population of voters; or at least the majority of them.
7CronoDAS
There are worse things that a politician can be than corrupt.
0[anonymous]
There are worse things that a politician can be than corrupt.
0Luke_A_Somers
Agreed. It's too easy to pander to a base that doesn't expect you to be good, just deliver a few things... things that matter a great deal less than the cumulative effect of having the right people in charge.

In Munich in the days of the great theoretical physicist Arnold Sommerfeld (1868–1954), trolley cars were cooled in summer by two small fans set into their ceilings. When the trolley was in motion, air flowing over its top would spin the fans, pulling warm air out of the cars. One student noticed that although the motion of any given fan was fairly random—fans could turn either clockwise or counterclockwise—the two fans in a single car nearly always rotated in opposite directions. Why was this? Finally he brought the problem to Sommerfeld.

“That is easy to explain,” said Sommerfeld. “Air hits the fan at the front of the car first, giving it a random motion in one direction. But once the trolley begins to move, a vortex created by the first fan travels down the top of the car and sets the second fan moving in precisely the same direction.”

“But, Professor Sommerfeld,” the student protested, “what happens is in fact the opposite! The two fans nearly always rotate in different directions.”

“Ahhhh!” said Sommerfeld. “But of course that is even easier to explain.”

Devine and Cohen, Absolute Zero Gravity, p. 96.

6Luke_A_Somers
So, uh, what's the explanation?
[-]Shmi100

The story appears to be apocryphal. I've heard many versions of it associated with various famous scientists. The source quoted is a collection of jokes, with very low veracity. Additionally, there are no independent versions of the story anywhere on Google. By the way, the quoted date of Sommerfeld's death is also incorrect. I wonder if there even were (unpowered) ceiling fans in Munich's trolleys during that time.

7Luke_A_Somers
Good point. Effects that don't exist don't need to be explained.
5Desrtopa
I'm not much of an engineer, but based on my understanding of their design from the description given, I can't see how they would even contribute to their alleged purpose.
0TrE
Perhaps because pressure is (approximately) constant, for every molecule going into the car, one must leave it (on average)?
0Luke_A_Somers
Trolleys have open windows in summer.
5John_Maxwell
It's an interesting story, but it might not be as silly as it sounds if one considers "ease of explanation" as a metric for how much credence one's model assigns to a given scenario. (Yes, I agree this is a hackneyed way of modeling stuff.)
6Eugine_Nier
Unfortunately, this seems to be the default way humans do things.
1John_Maxwell
Well, the world is a complicated place and we have limited working memory, so our models can only be so good without the use of external tools. In practice, I think looking for reasons why something is true, then looking for reasons why it isn't true, has been a useful rationality technique for me. Maybe because I'm more motivated to think of creative, sometimes-valid arguments when I'm rationalizing one way or the other.
[-]philh620

Men in Black on guessing the teacher's password:

Zed: You're all here because you are the best of the best. Marines, air force, navy SEALs, army rangers, NYPD. And we're looking for one of you. Just one.
[...]
Edwards: Maybe you already answered this, but, why exactly are we here?
Zed: [noticing a recruit raising his hand] Son?
Jenson: Second Lieutenant, Jake Jenson. West Point. Graduate with honors. We're here because you are looking for the best of the best of the best, sir! [throws Edwards a contemptible glance]
[Edwards laughs]
Zed: What's so funny, Edwards?
Edwards: Boy, Captain America over here! "The best of the best of the best, sir!" "With honors." Yeah, he's just really excited and he has no clue why we're here. That's just, that's very funny to me.

8juped
The scene in question.
3DSimon
That whole testing sequence is one of the best examples in film of how to distinguish what's expected of you from what's actually a good idea. (Or in that specific case, what seems to be expected of you.)

It is because a mirror has no commitment to any image that it can clearly and accurately reflect any image before it. The mind of a warrior is like a mirror in that it has no commitment to any outcome and is free to let form and purpose result on the spot, according to the situation.

—Yagyū Munenori, The Life-Giving Sword

Shipping is a feature. A really important feature. Your product must have it.

-Joel Spolsky

Real artists ship.

-- Steve Jobs

(The Organization Formerly Known as SIAI had this problem until relatively recently. Eliezer worked, but he never published anything.)

8cody-bryce
And they ship the characters the fans want.
6fubarobfusco
If your service is down, it has no features.
6DanArmak
And no bugs.
6ygert
Well, there is one pretty major bug: That your service is not doing anything at all!
8fubarobfusco
It has all the bugs. All of them. (Well, not really. For instance, it doesn't have any security holes.)
2Strange7
If it bears any resemblance to a product at all, your own admin-level access constitutes a potential security hole.
8Shmi
It's a feature.
3A1987dM
I would have quoted more, because on reading that out of context I was like “YOU DON'T SAY?”

Most people, when giving advice, don't optimize for maximal usefulness. They optimize for something like maximal apparent-insight or maximal signaling-wisdom or maximal mind-blowing, which are a priori all very different goals. So you shouldn't expect that incredibly useful advice sounds like incredibly insightful, wise, or mind-blowing advice in general. There's probably a lot of incredibly useful advice that no one gives because it sounds too obvious and you don't get to look cool by giving it. One such piece of advice I received recently was "plan things."

6Nornagest
There's probably also a lot of useful advice that our minds filter out because it scans as obvious or trivial. Even when I'm trying to give maximally effective advice, I usually spend a lot of effort optimizing it for style; the better something sounds, the more people dwell on its implications and the likelier it is to stick. Fortunately, most messages leave plenty of latitude for presentation. Alternately, you could try dressing simple advice up in enough cultural tinsel that it looks profound, as suggested here.
7Stabilizer
Well, a lot basic rationality literally seems to be about doing what is almost obvious but is hard to do because of bugs in your cognitive architecture. This reminds me of the following quote by Elon Musk in an interview where he was asked what he would say to new start-up founders:
0Richard_Kennaway
And by the same author: and (because what counts after getting it out the door is how many people actually use it.)
2pjeby
That's Jeff Atwood. The quote is from Joel Spolsky. While the two both work together on Stack Exchange, they're different individuals.

I do not love the bright sword for its sharpness, nor the arrow for its swiftness, nor the warrior for his glory. I love only that which they defend

Faramir, from Lord of the Rings on lost purposes and the thing that he protects

2Dorikka
Except that a non-overwhelming love of a useful art may help you become better in the art, even though you would switch to another if it helped you optimize more.
-2hankx7787
another great quote for 2013

Things that are your fault are good because they can be fixed. If they're someone else's fault, you have to fix them, and that's much harder.

-- Geoff Anders (paraphrased)

3Giles
Did he mean if they're someone else's fault then you have to fix the person?
3Qiaochu_Yuan
Yep.

You want accurate beliefs and useful emotions.

From a participant at the January CFAR workshop. I don't remember who. This struck me as an excellent description of what rationalists seek.