This is our monthly thread for collecting these little gems and pearls of wisdom, rationality-related quotes you've seen recently, or had stored in your quotesfile for ages, and which might be handy to link to in one of our discussions.

  • Please post all quotes separately, so that they can be voted up/down separately.  (If they are strongly related, reply to your own comments.  If strongly ordered, then go ahead and post them together.)
  • Do not quote yourself.
  • Do not quote comments/posts on LW/OB.
  • No more than 5 quotes per person per monthly thread, please.

 

Rationality Quotes: July 2010
New Comment
227 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]Kutta440

If anything of the classical supernatural existed, it would be a branch of engineering by now.

-- Steve Gilham

8JohannesDahlstrom
The Salvation War Web Original trilogy is based on this premise. And boy does it makes good use of it.
2simplicio
I've just been reading it. Oh dear... this series looks to be fun and really really bad. "Good Omens" (Pratchett and Gaiman) is an excellent book along such apocalyptic lines.
0apophenia
I've been finding it to be rather terrible, in terms of plausibility. I won't rip into it at length, since the tvtropes link seems to do a decent job.
4gwern
Relevant xkcd: http://www.xkcd.com/808/
3SilasBarta
Now that you mention it, this is kind of the premise of the Star Ocean series. It has "symbology" (as it's called in the third installment, Till the End of Time), which is basically the ability to manipulate nature to do "magical"-seeming things by formation of particular symbols. The people in these worlds harness this capability for standard engineering purposes: they build air-conditioning units that draw their coldness from application of specific symbols. The plot of End of Time revolves around a professor combining symbological powers with those of genetics.
2Leonhart
TTEoT is a superb game, and that's not the only LW-relevant theme it contains. Spoilers: Gur cynlre punenpgref ner NVf jub cebprrq, va gur pybfvat npgf bs gur tnzr, gb rfpncr gurve obk. Gur raqvat vf n Crezhgngvba Pvgl-yvxr fpranevb jurer gur "birefrref" fuhg qbja gur jbeyq-fvzhyngvba - ohg vg pbagvahrf naljnl qhr gb vagreany frys-pbafvfgrapl.
6[anonymous]
So it's the same as the plot of Harry Potter and the Methods of Rationality? >.>
0Desrtopa
Oh god, I hope not. You can see in my above comment what that would actually entail.
0Desrtopa
As it was presented in the game, I personally found that to be among the worst plot twists I had ever experienced. Fnlvat gung gur znva punenpgref ner nyy NVf fbhaqf n ybg pbbyre guna gur zber cerpvfr eriryngvba, gung gurl'er nyy ACPf sebz na ZZBECT cynlrq ol orvatf sebz "4 qvzrafvbany fcnpr" juvpu gheaf bhg gb or culfvpnyyl nyzbfg rknpgyl yvxr gurve bja havirefr.
0Leonhart
Oh, hush you. With my LessWrongian superpowers, I know what the game was meant to be about. It's not our fault that the scriptwriters just had to be rephrased with the "sbhegu qvzrafvba" herpaderp so that lesser mortals could follow it. :) (I'm now imagining a MethodsOfRationality!Fayt who had been expecting the plot twist based on ubj fhfcvpvbhfyl uhzna nyy bs gur "nyvra" enprf va gur tnynkl jrer...)
0Will_Newsome
:D I was rather disappointed in Eliezer for not giving Star Ocean: Till the End of Time a mention in his ultimate crossover fanfic. My cat is named Fayt. (The other one's named Lyra.)
3Peter_de_Blanc
It does, and it is.
2Blueberry
Are you referring to psychology?
4Peter_de_Blanc
I posted without having thought of any examples, still confident that the statement is true.
4James_K
The nearest example I can think of is the alchemical concept of transmuting one metal into another. This process is of course central to nuclear reactions, but pre-20th Century was considered physically impossible.

Lightning was the weapon of Zeus. Now it can be controlled by electrical engineers.

The Aztecs thought the sun was a god. Now plasma physicists can produce light via similar means.

3James_K
Those are good examples.
1Emile
Magnetism?
0fortyeridania
People have cited some very modern inventions, but ever since the first invention, people have been inventing things that were previously relegated to magic. I mean, what else is technology?
6gwern
Some technology is for replacing things ascribed to magic. But most technology is fulfilling needs that a wizard doesn't know he has, or is there to solve a need or problem that wouldn't except for some other bit of technology. Look at a few thousand (out of the millions of existing) patents. How many of them have a direct analogue in magic such that their utility really is something 'previously relegated to magic'? - not vaguely falling in a general class of functionality which you once saw a spell kinda-sorta like. For example. Recent news about an environmentally friendly rocket fuel made out of just nitrogen and oxygen. This is a very modern invention. It's very useful. What magic does it replace? 'Oh, rocket fuel replaces flying on a broomstick!' Ah, but I wasn't talking about rocket fuel in general and certainly not rockets+rocket-fuel, because this invention was not of 'rocket fuel in general'. This invention was of a particular kind of rocket fuel. What magic does this particular kind of rocket fuel replace? The answer is none. Flying magic is conceived of as intrinsically environmentally friendly. Harry Potter never asks himself how he will pay for the bioremediation of the hydrochloric acid left in the wake of his broomstick. Take a look at Kevin Kelly's discussion of sparkcatchers. What magic is this technology replacing? This isn't even particularly esoteric stuff like I could start pulling out from biology or chemistry. (What wizard - or author - ever had the imagination to think of bacteria glowing green like fireflies?) This kind of statement reminds me of an upvoted quote here somewhere which went 'since when has the majority ever been right about anything?' This irritates me, because the majority is right. Of the infinite class of propositions the majority holds, most are right, and the ones the majority has been wrong about, like geocentrism, have approximately measure 0. It's right about everything else. The majority in the room I am in think
1soreff
Very much agreed. By the way, this one one of the points W. Brian Arthur makes in The Nature of Technology which I recommend as a well written exposition of quite a number of the dynamics in technology development. One very broad range of technology which has no analog to magic is high precision instrumentation. There are very few situations in daily life when I need to know a measurement to within 0.1%, and they are many technological and scientific situations where that knowledge is necessary.
1gwern
Actually, Arthur is already on my reading list. ;) I think I must have seen him recommended before by Kevin Kelly or perhaps Don Ihde. I was going to point out that another good example is all the sensory modalities that science knows of, like electromagnetism in all its forms and frequencies, or gravity itself, but I thought that they were too arguably close to magical 'second sight' or chi skills like sensing someone nearby.
0shokwave
You have to be careful that you don't go from "the majority is right in most cases" to "the majority is right in this case" without good reason. Prior to any information about this case, sure, it's a good guide. But if you have reason to believe that this case isn't a trivial inference from overwhelming direct observational evidence, then the majority is much less of a guide. When it comes to a socially-accepted conclusion, a severe dearth of evidence, and motivations to hold a certain point of view, 70-90% - the majority - get it wrong. On the phrase "since when has the majority ever been right about anything?": on most points of difference, the majority has been wrong first and right only later. It is easy to look at only the points of difference: when it comes to points of similarity, you are part of the majority, and so you're not really interested in knocking them down. Perhaps a better formulation would be "since when has the majority ever been right about anything that isn't simultaneously important and obvious ?" The problem with saying is that most are boring. On the class of interesting propositions, the majority does not have a good track record.
0[anonymous]
Voted down because of epistemic strutting. Not citing examples in your first comment is one thing, but you could at least have done so in your follow-up.

Doubt, n. The philosophical device Descartes so cleverly used to prove everything he previously believed.

-L. A. Rollins, Lucifer's Lexicon

0wedrifid
Wow. I'm going to have to get myself a copy of that book!
[-]Rain320

Nature draws no line between living and nonliving.

-- K. Eric Drexler, Engines of Creation

Hunches are not bad, they just need to be allowed to die a natural death when evidence proves them wrong.

-- Steve Moore, former FBI agent

From the Wikipedia article about perverse incentives:

In Hanoi, under French colonial rule, a program paying people a bounty for each rat pelt handed in was intended to exterminate rats. Instead, it led to the farming of rats.

and

19th century palaeontologists traveling to China used to pay peasants for each fragment of dinosaur bone (dinosaur fossils) that they produced. They later discovered that peasants dug up the bones and then smashed them into multiple pieces to maximise their payments.

6Theist
[context added] -- Soul Music, by Terry Pratchett
1mattnewport
Needs more context.
[-]CSmith240

"Convictions are more dangerous enemies of truth than lies."

--Friedrich Nietzsche

"I am free, no matter what rules surround me. If I find them tolerable, I tolerate them; if I find them too obnoxious, I break them. I am free because I know that I alone am morally responsible for everything I do."

Robert A. Heinlein

The most exciting phrase to hear in science, the one that heralds new discoveries, is not "Eureka!" (I found it!) but "That's funny..."

-- Isaac Asimov

3gwern
Repeat?
9Richard_Kennaway
I only judge something as a repeat if it's already appeared in a quotes thread. Sometimes it's worth pulling out something quoted in the middle of a discussion and presenting it here.
1Kaj_Sotala
Apparently, yeah. Not in a quotes thread, though.
[-]torekp190

Idealists of all schools, aristocrats and bourgeois, theologians and physicians, politicians and moralists, religionists, philosophers, or poets, not forgetting the liberal economists - unbounded worshippers of the ideal, as we know - are much offended when told that man, with his magnificent intelligence, his sublime ideas, and his boundless aspirations, is, like all else existing in the world, nothing but matter, only a product of vile matter.

We may answer that the matter of which materialists speak, matter spontaneously and eternally mobile, active, productive, matter chemically or organically determined and manifested by the properties or forces, mechanical, physical, animal, and intelligent, which necessarily belong to it - that this matter has nothing in common with the vile matter of the idealists. The latter, a product of their false abstraction, is indeed a stupid, inanimate, immobile thing, incapable of giving birth to the smallest product, a caput mortuum, an ugly fancy in contrast to the beautiful fancy which they call God; as the opposite of this supreme being, matter, their matter, stripped by that constitutes its real nature, necessarily represents supreme nothingness.

--Mikhail Bakunin, God and the State

Reminded me of some posts here by Academician.

2MichaelVassar
I wish I could upvote this, maybe 10 or 20 times. It's essentially a concise statement of my religion and I'm putting it on Facebook today.
[-]Rain190

The mind is not a vessel to be filled but a fire to be kindled.

-- Plutarch

[-]josht180

A recent one from Linux Weekly News that gives insight into rationality:

Side note: when a respected information source covers something where you have on-the-ground experience, the result is often to make you wonder how much fecal matter you've swallowed in areas outside your own expertise. -- Rusty Russell

2stcredzero
Similar to what professor Karel Culik said about Time magazine. He thought Time was really good, until he read something about computer science, and he started to wonder about the rest of it.
0HonoreDB
This was the original sense of TVTropes/DanBrowned, although the scope seems to have broadened a bit.

I was gratified to be able to answer promptly, and I did. I said I didn’t know.

-- Mark Twain, "Old Times on the Mississippi"

[-]Alan170

What frightens us most in a madman is his sane conversation.

--Anatole France

6gwern
I can't find any source for this, so it may be apocryphal.
[-]Rain170

A superstition is a premature explanation that overstays its time.

-- George Iles

"Anyone who believes that the theory of evolution implies moral darwinism, and who also believes in the theory of gravity, has a moral duty to go jump off a cliff." -- Ari Rahikkala

2JenniferRM
Is there anyone who actually believes in moral darwinism under that self-description, or is it just a straw man position that people like to claim as the underpinning of the ideology of people they don't like? When I tried to google the term I found the book Moral Darwinism: How We Became Hedonists which is described thusly in a five star review:
6Tyrrell_McAllister
I think that the quote is best read as a critique of any appeal to nature. Maybe you don't see such brazen appeals to nature in careful arguments, but they seem common enough in informal arguments. Many people really do seem to think that you've made a substantive point about what we ought to do when you point out that something happens "in nature". For example, I knew someone once who was uncomfortable with tolerating homosexuality, but who felt obliged to seriously reconsider his position when he learned that animals sometimes exhibit homosexual behavior. I was in the strange position of trying to explain why he ought not to be persuaded by that argument, even though it led him to what I thought was the right conclusion.
1Blueberry
Did you then tell him about chimpanzee war, murder, and infanticide? The appeal to nature is very common, and I'd rather get rid of that than change someone's mind about same-sex behavior, which seems to be a terminal value difference anyway.
5SilasBarta
So, Wiker thinks that the eventually-dominant (?) Epicurean movement staged a deliberate 2-millennium (?) campaign to get a 60s-style social revolution (?) going, and thereby steal credit (?) for scientific advancement from the Church, which was doing so well at it (?) for the first 1500 years. Before reading it: what are the odds Wiker has marshaled enough evidence to even get that hypothesis on the radar? Oh, and check this out from the description: Yeah, we sure don't have any evidence of a Greek culture practicing infanticide, do we? Edit: Also, the review you refer to is by a "Phillip Johnson", who looks to be this Phillip Johnson, founder of the Intelligent Design movement.
2JenniferRM
I think that you're agreeing with me? My point was that Wike clearly has an axe to grind and was attributing belief in "moral darwinism" to people who don't even know what it means. You're logically taking apart the theory I was quoting... thus we're in agreement... or not? Or something. The thing I was puzzled by was the way the original quote had a surface layer of congruence (darwinism, suicide for stupid people, etc) but was difficult for me to coherently parse when I tried to work out the implications and examine the logic that inspired it. I came up wanting when I did that, and that lead me to question whether the ideas themselves were well grounded. If not, where did the ideas even come from?
2SilasBarta
Yes, we're in agreement. I was just shocked by how bizarre the claim is, given how many improbable pieces he's stitched together. I think Johnson -- or Wiker, if that's what the book really argues -- came to that belief by fitting his own view sympathetically into a larger narrative, and found it convenient to stretch it as far back in time as possible. In fairness, when reading about Epicurus, a lot of his ideas do match modern post-60s beliefs, but there isn't a common demographic that endorses the whole package.
[-]DSimon160

When I was 14, my father was stationed in Japan. I went rock climbing with this kid from school. He fell and got injured, and I had to bring him to the hospital. We came in through the wrong entrance, and passed this guy in the hall. He was a janitor. My friend came down with an infection, and the doctors didn't know what to do. So they brought in the janitor. He was a doctor. And a Buraku - one of Japan's untouchables. His ancestors had been slaughterers, gravediggers. And this guy knew that he wasn't accepted by the staff, didn't even try. He didn't dress well. He didn't pretend to be one of them. People around that place didn't think he had anything they wanted, except when they needed him - because he was right, which meant that nothing else mattered. And they had to listen to him.

-- Dr. Greg House

[-]i77120

"We are selfish, base animals crawling across the earth. But because we got brains, if we try real hard, we may occasionally aspire to something that is less than pure evil."

-- Gregory House

1sark
Once we have achieved unerring correctness in all domains obscure or not, all we have to do is contrive a situation in which our correctness is useful and in which others will require our knowledge. Then we will be respected.
3DSimon
Yeah, I really thought that was true at some point... but as it turns out, being good at fixing broken Windows computers hasn't made me respected, just commodified. :-)
0RHollerith
That's probably because the people whose computers you fix do not understand how difficult what you do is -- or if they do, they also understand that another way to fix it is to reinstall the operating system.
[-]brazzy160

The necessity for marking our classes has brought with it a bias for false and excessive contrast, and we never invent a term but we are at once cramming it with implications beyond its legitimate content. There is no feat of irrelevance that people will not perform quite easily in this way; there is no class, however accidental, to which they will not at once ascribe deeply distinctive qualities. The seventh sons of seventh sons have remarkable powers of insight; people with a certain sort of ear commit crimes of violence; people with red hair have souls of fire; all democratic socialists are trustworthy persons; all people born in Ireland have vivid imaginations and all Englishmen are clods; all Hindoos are cowardly liars; all curly-haired people are good-natured; all hunch-backs are energetic and wicked, and all Frenchmen eat frogs. Such stupid generalisations have been believed with the utmost readiness, and acted upon by great numbers of sane, respectable people. And when the class is one's own class, when it expresses one of the aggregations to which one refers one's own activities, then the disposition to divide all qualities between this class and its converse, and to cram one's own class with every desirable distinction, becomes overwhelming.

-- H.G. Wells, A Modern Utopia

Silas will like this one:

Menahem sighed. 'How can one explain colours to a blind man?'

'One says', snapped Rek, 'that red is like silk, blue is like cool water, and yellow is like sunshine on the face.'

-- David Gemmell "Legend"

I was actually starting another article that presents a solution (well, a research program) for qualia. [1] The idea is this:

The concept of qualia becomes mysterious when we have a situation in which sensory data (edit: actually, cognition of the sensory data) is incommensurable (not comparable) between beings. So the key question is, when would this situation arise?

If you have two identical robots with idential protocols, you have no qualia problem. They can directly exchange their experiences and leave no question about whether "my red" is "your red".

But here's the kicker: imagine if the robots don't use identical protocols. Imagine that they instead simply use themselves to collect and retain as much information about their experiences as physically possible. They optimize "amount I remember".

In that case, they will use every possible trick to make efficient use of what they have, no longer limited by the protocols. So they will eventually use "encoding schemes" for which there is no external rulebook; the encoding is implicitly "decompressed" by their overall functionality. They have not left a "paper trail" that... (read more)

9JenniferRM
Before finishing (or perhaps as a sequel?) you should make sure to watch Cristof's Koch's "neural correlates of consciousness" talk. He's been giving variations on this talk for something like 10 years that I know of and its pretty polished. Its gotten better over the years and the speaker is the source of my current working definition of consciousness (quoted below). Which is not about language I/O and compression but about internal experiences themselves and what systems implement them. The core insight is that you can show someone a visual trick (like the faces or goblet image) and you can go back and forth "seeing different interpretations". When you're in one or the other state "internal state" this is you having different kinds of "qualia", and presumably these distinct perceptual states have "biological correlates". Manipulation of these internal mental states and study of the associated physical systems become the "object of study" in order to crack the mind-body problem. Once you've got the neural level you can ask about high level issues like algorithms or ask about deeper mechanisms like neurotransmitters and genes and so on. Full understanding would imply that we could create mutant "zombie mice" and that they would have no qualia (of certain sorts) and be incapable of whatever behavior was "computed" in a way that involved (that sort of) qualia. Ideally we would have a theory to predict and and explain such phenomenon and perhaps we'd be able to invent things like a pill that lets you "become a p-zombie" for an hour (though I suspect part of that would involve shutting down enough memory formation processes that you would not be able to remember the experience except via something external like videotape). The Q&A has much more sophisticated objections/questions than you usually get on the subject of minds. The final question ends with Koch's working theory which sounds about right to me. Quoting Koch when asked why he thinks bees are probably consc
3SilasBarta
Thanks for the pointer! This will help me to connect my speculations to the existing literature. Any text version/transcript of this lecture, or paper that explains the points in the talk?
0JenniferRM
My summary above cuts to what I think is the core insight about focusing clearly and experimentally on exactly the elements of interest: consciousness and neurons. The talk itself is a summary of the main points of many papers with different points and some demonstrations of the experimental manipulations. The talk itself (rather than the intros) starts about 4 minutes in. I recommend just watching it. Koch is a pretty good speaker and this is sort of his "dog and pony show" where he summarizes an entire research program in a way that's been iteratively optimized for years. Your 60 minutes will not be wasted :-)
1Oscar_Cunningham
Up voted, looking forward to you posting some of these in-progress articles. Should: Be vice-versa?
0SilasBarta
Thanks! And yes it should, I'll correct that.
0WrongBot
This seems to match up with an argument I made against writing oneself into the future, though I think your formulation is more general. I'm certainly quite interested in hearing more on the subject; I think you're headed in a good direction.
4SilasBarta
Thank you much, and I definitely see the similarity with what you posted. I may indeed be running into the problem of letting "good enough" become the enemy of "at all". I'll try to get these articles up in some presentable form soon.
2cousin_it
The drafts you've been posting lately are interesting to me, and I'd like to see them fleshed out into top-level posts. I would also suggest adding external material and references to give more context to your thought experiments.
0SilasBarta
Thanks. But that's one of my difficulties. I read a lot of stuff and so these ideas just "come together". I don't even know if there is a source that agrees with this idea. As it stands now, the only sources I believe I'd be able to cite are some of Gary Drescher's discussion of qualia, and the information-theoretic basics of how compression works, and what makes it more or less effective. Any suggestions (specific suggestions) for how to find the external references that would be relevant to this topic or the other one's I've posted recently? By the way: I started keeping a list of planned top-level articles on my Wiki page.
1NihilCredo
I 'get' the other two, but the red-silk analogy eludes me.
2Richard_Kennaway
I have heard of a blind man comparing red (as he had heard of it) to the sound of a trumpet.
0NihilCredo
Hmm... can a blind man possess synæsthesia? It seems possible if the source of the blindness is in the eye rather than the brain.
4Jonathan_Graehl
More likely color-words as concepts exist in the brains of blind people because there are interestingly different distributions of other directly meaningful concepts associated in proximity with others' usage of the color-words, and with actual objects in the world (presuming that their color is sometimes described to you). I think Richard put this very well when he said "as he had heard of it".
2Blueberry
Probably. A color-blind person can.
1Blueberry
Me too. Silk is more like beige or cream color. How can you describe red without talking about fire?
3KrisC
Or this infographic from Information is beautiful. The texture of silk is smooth and easily slips from the grasp. It is exciting and sensuous as a fabric. Anger, courage, love, heat, and passion, according to the chart. Perhaps it might be easier to read them the referenced chart's correspondences.
9twanvl
That infographic would have been much better as a regular table, instead of this circular thing. It seems to me as if it is not intended to be actually used, only to look nice.

It's a dreadful graphic. No information leaps out at the viewer, you have to hunt through two tables for the meanings of the letters and numbers. It takes an effort to find the letter for any given block, or to find the block for any given letter, in radii far from where the letters appear. It's difficult to tell apart yellow and gold, or grey and silver: the key only serves to highlight how indistinguishable the colours are.

And since this graphic does not work, I cannot see it as beautiful. It is an ugly sacrifice of function to superficial prettiness.

1cupholder
Agreed. I wish they'd stick to calling hard-to-read graphics like this 'visualizations' - the word 'infographics' implies a graphic designed to efficiently display information. The worst part is it wouldn't be hard to improve the graphic. They could drop the annoying 84-item list and just directly write the emotions in the 84 slots around the circle instead of using numbers. Enlarge the circle and blow up the font size a bit - then they can put the A to J list of cultures into the empty middle of the circle so you don't have to keep looking off the side to cross-reference it. That'd help, even if it wouldn't fix it. Edit - I see that when they used that infographic as their book's cover, they gave up on the idea of making it a real infographic and just made it into a pretty flower!
0Blueberry
So looking at numbers 31, 2, and 46, a friendly AI should be orange and blue... I don't associate red with smoothness, or silk with any of those emotional qualities.
2wedrifid
Are you sure about that? Your emotions don't ask you for permission before they make associations.
-1Blueberry
I can't tell if this is wordplay based on the ambiguity in the quoted sentence (in which case, I like the joke :) ) or if you're serious. If you're serious, then yes, I'm sure: while emotions may not ask me for permission, I'm aware of what they're associated with.
0red75
Strange. I have associations for green (foliage, leafs), yellow (sun), blue(water). Red requires conscious effort to select domain for association to pop up.
-7[anonymous]
2JenniferRM
Or blood?

This is what fascinates me most in existence: the peculiar necessity of imagining what is, in fact, real.

-- Philip Gourevitch

[-]djcb130

Their judgment was based more upon blind wishing than upon any sound prevision; for it is a habit of mankind to entrust to careless hope what they long for, and to use sovereign reason to thrust aside what they do not fancy.

-- Thucydides, Greek Historian, ca. 5th century BCE (Book IV, 108)

I like Thucydides for the way he tries to explain history in terms of real-politik, people, their drives and especially without including the gods in an explanation, somewhat similar to Hippocrates.

Interestingly, a modern version of this appeared in Neal Stephenson's Anathem:

Never believe a thing simply because you want it to be true

where it's called Diax's Rake.

Anathem is a great book, I'd like to add, and quite well aligned with many of the LW themes.

2gwern
No love for Wizard's First Rule?
0[anonymous]
I have to disagree on two counts. First, Diax's Rake is explicitly a reference to Thucydides, (or, more spoilerifically, Thucydides' "referenced" Diax for some value of reference), so it's not really interesting that it appeared in Anathem. Secondly, Anathem isn't actually aligned with LW themes at all. It might appear that way at the beginning, but Stephenson undoes all of it with the spoiler twist at the end.
0djcb
Hmm.. i can't remember the specific reference to Thucydides from the book, and I don't have it handy right now... did the book mention him? I just found the parallel quite interesting. Regarding the other point, I meant 'aligned with LW themes', that is discusses many of the same things that are discussed here -- and in many cases seems to agree. Not always - but apart from the parallel universe mixups which are a bit... suspect, I got the idea that NS has been lookin at LW (well, OB) and similar sources.

A man should never be ashamed to own that he has been in the wrong, which is but saying in other words that he is wiser today than yesterday.

Jonathan Swift (also attributed to Pope)

I do not think much of a man who is not wiser today than he was yesterday.

Abraham Lincoln

2xamdam
=> A man who is not ashamed of being wrong will be thought of much by Abraham Lincoln? As an aside, one should be ashamed of being wrong when there was sufficient time and information to be right, and still admit being wrong. It should lead to better care taken next time.
3Blueberry
You should resolve to take better care next time, and evaluate where your mistake was, but I don't see how shame helps anything.
2xamdam
Getting singed helps avoid hot objects. Shame helps avoid stupidity.

Shame leads to a variant on guessing the teacher's password-- an effort to not piss people off, without asking them what might be problematic. After all, you're supposed to know better than to make that mistake.

3xamdam
"The shameful does not learn" - Talmud I agree with your point, but I do not think the emotion should be dispensed with altogether, if that's really possible. I think you can separate social embarrassment, which in this context is counterproductive, from deep very personal private embarrassment of having been stupid.

For an even somewhat rational person, pain is far stronger than necessary as a warning sign. As someone generally concerned with my own body's welfare, the mental equivalent of popping up a politely worded dialog box would be sufficient. I find that shame is likewise overkill for solving this problem.

5rwallace
Personally, I don't mind pain being a strong enough warning that it's hard to ignore. I can see the need for that. I think the problem with pain is that it's like those stupid car alarms. You know the ones that should be programmed to turn off after five minutes because any good they might do will have been done by then if it's to be done at all, but they actually keep going all night? That's what pain should have: a way of saying, after some appropriate enforced time delay, fine, I've got the message, I'm doing everything I can about the problem, you can stop now.
4SilasBarta
I can't. I've had problems with pains that are demonstrably unrelated to any threat to bodily integrity and for which there is no known technique that removes it. If pain were limited to real threats, I'd agree, but it's not. So it's not even an issue of "yeah, I get the message, you can stop reminding me"; often times, there is no message to be given, just suffering.
3Blueberry
Have you never been tempted to push ahead doing something you want, and ignored a minor pain? We'd just ignore pop-up boxes if we were in the middle of something we considered important.
0WrongBot
Absolutely. But this is not a way in which pain is superior to a pop-up box. If the pop-up box that replaced intense pain had alarms and flashing lights attached, and the one for more minor pains did not, I would pay attention to the alarms and flashing lights.
[-]DSimon140

Speaking in terms of real pop-up boxes, you might be surprised at how easy it is for people to ignore the content of even the most blaring, attention-grabbing error messages.

A typical computer user's reaction to a pop-up box is to immediately click whatever they think will make it go away, because a pop-up box is not a message to be understood but a distraction from what they're actually trying to accomplish. A more obnoxious pop-up box just increases the user's agitation to get rid of it.

As rationalists, we try hard to avoid falling into traps like these (I'm not sure if there's a name for the fallacy of ignoring information because it's annoying, but it's not exactly a high-utility strategy), but part of the way we should do that is to design systems that encourage good habits automatically.

I like Firefox's approach; when it wants you to choose between Yes or No on an important question ("Really install this unsigned plugin?"), it actually disables the buttons on the pop-up for the first 3 seconds. You see the pop-up box, your well-honed killer instinct kicks in and attempts to destroy it by mindlessly clicking on Yes so you can get back to work already... but that doesn't work, you're surprised, and that jolt out of complacency inspires you to actually read the message.

I suspect a "Hey, have you noticed that something has penetrated the skin of your left foot?" warning might benefit from having the same mechanism.

5WrongBot
I appear to be a mutant: I always read pop-up boxes. By all means, please adapt my analogy to something that you would actually pay attention to.
3sfb
That doesn't seem a lot better - I know what it wants me to do with an unsigned plugin prompt - review the site, verify the download hash, review the plugin source code, look for reviews of the plugin, author, site... What I actually do is wait 5 seconds, then click "yes do it" as soon as possible. So the utility is ... well intentioned, but still ineffective.
1DSimon
It's true that it's less than perfectly effective, but it serves some purpose: I almost always install plugins from the Mozilla plugin site, where a rating is immediately available, and where a virused plugin would probably get removed very quickly. Under those conditions, I know that I'm fairly safe just installing it anyways. However, a malicious site could attempt to infect my browser by installing a plugin, which is where the timer comes in handy. It could even attempt to hide the plugin dialog with lots of other useless dialogs ("Really submit this comment?" Yes. "Really really submit this comment?" Yes. "Really really REALLY submit this comment?" Yes. "Install this plugin?" Yes. Oh, hold on, wait! Crap.) More generally, timed dialogs are helpful because they increase the chance that you notice what it is you're confirming. If you know you're doing something risky and want to do it anyways, so be it... but at least you know what it is you're accepting, and are given a greater opportunity to back out if you are surprised by the level of risk.
1Oscar_Cunningham
The about:config option "security.dialog_enable_delay" allows one to reduce the delay to 0.
0Theist
It seems pretty clear to me that this was not true in our ancestral environment. It may be the case in our present artificially benign environment however.
1WrongBot
That is precisely what I mean; but also note that there are circumstances in the ancestral environment in which pain is entirely useless, such as when one has been mortally wounded. So even in the worst case we can do better than pain, and in the current case I suspect we can do much, much better.
1SilasBarta
Evolution has the problem of path-dependence, though. Once the "don't do that" / "pay attention to that" mechanism builds up slowly over many generations, it cannot refactor in such a way that it surgically cuts out the internal feeling of pain in precisely those circumstances where, "hey, might as well give up".
0mattnewport
It's hard to see what reproductive benefit there would be to reduced suffering when dying either so there is unlikely to be any evolutionary pressure in that direction.
0xamdam
Thanks for the explanation. I like the argument, but still willing to play the devil's advocate: popup box is nice when you're paying attention, but it does not produce learning. Imagine working on a plant where a wrong move can cause a serious injury. Popup boxes will not produce the muscle memory needed to navigate. I would argue this strongly about pain, but I am not sure how well the analogy transfers to mental errors.
2WrongBot
Well, it would have to be a popup box that interrupts whatever you're currently thinking about. It grabs focus, to continue the analogy. Does pain produce muscle memory? I haven't heard that before.
1xamdam
Maybe a popup would be better, but until we can hack our brains the question remains whether you're better off with shame as a learning mechanism. Does pain produce learning? Probably yes. http://www.nytimes.com/1991/08/27/news/pain-and-learning-may-be-close-cousins-in-chain-of-evolution.html?pagewanted=all
1WrongBot
To continue the computer analogy, it would have to be a popup box that steals focus, so that you can't do anything else until you acknowledge it's there. Does pain aid in creating muscle memory? I hadn't heard that before.
1red75
Should it pop up again if pain increases from mild to strong? Should it pop up periodically when pain is extreme? I'd rather prefer to consciously disable drive to remove pain source, but stay informed of kind and intensity of pain. Edit: AFAIK insects use that kind of pain processing, they react on pain, but they don't get overwhelmed by it.
1WrongBot
Sure. The analogy to computers is not a perfect one, because brains don't function like modern computer operating systems. My objection to pain is not that it is uninformative, it's that it's overwhelmingly unpleasant even when we do not wish it to be.
1red75
Yes, it can inflict more harm by forcing you into suboptimal decisions. Shame can be alike too. So, I vote for insect-like shame and pain processing, I've mentioned in grandparent.
1sclamons
There are cases of people with no sense of pain. Here's an article about it: http://findarticles.com/p/articles/mi_m1200/is_25_170/ai_n26713199/ ...and a link to the primary it references, if you have a subscription to Nature: http://www.nature.com/nature/journal/v444/n7121/full/nature05413.html A relevant quote from the review: "The team's first research subject, a 10-year-old boy, was well known in his community for street performances in which he placed knives through his arms and walked on hot coals. Despite tissue damage, he apparently felt no discomfort." Other mutants for the same gene were similar -- they generally enjoyed showing off their lack of pain by deliberately injuring themselves. One killed himself by jumping off a roof, I believe just after the above article was published (sorry, I'm not really sure where I read that). Living without pain is generally an unhealthy idea. Without visceral negative feedback, humans don't place value on bodily integrity and tend to self-terminate, which is probably why those mutants are incredibly rare. Perhaps once aversion reactions have been formed, knocking out pain receptors would be a good thing. I'd certainly be curious to see if people still avoid injuring themselves if they lose the ability to feel pain AFTER they've experienced it all their lives.
3Cyan
They don't. People who develop pain asymbolia from various insults to the brain are not strongly motivated to avoid pain. (And these are people who have functioning nociception, unlike the family described in the article you link.)
2WrongBot
Yeah, knocking out pain receptors in children seems like a very bad idea. Pain is unnecessarily harsh if you already have a strong interest in preserving your body; then all you really need is to know that you're doing something bad to yourself, and that's more than enough reason to stop. But for those who are not yet sufficiently rational, like (most) children, it's probably not something to be messed with without serious consideration. It sounds like that kid has some of the worst parents in human history. How do you let your kid get a reputation for self-mutilation and not, y'know, stop him?
2wedrifid
Stop your child from getting status for himself and your family? Inconceivable!
0red75
Knocking out nociceptors? It's not what I thought about. Knocking them out reduce available infomation to make decisions. I meant deliberate suppression of goal-shifting and brain resource allocation effects of nociceptors activity (in a sense proposed by Marvin Minsky). It is hard to evaluate effects of this change on child's pain aversion behavior, but I can hypothesize that this kind of pain control could be ineffective until "central executive" is sufficiently developed. Edit: In the case of mild pain this way of dealing with it can be exercised on our current brainware.
3apophenia
I disagree--perverse incentives. If I was ashamed every time I was wrong, I might be more careful. Or, I might stop admitting I was wrong. I make an effort to congratulate myself for admitting I'm wrong, for this reason. Now that I think about it, it would be even more helpful to find out which of these I would do.
2[anonymous]
For most people, it mostly makes them avoid admitting their stupidity.
0[anonymous]
You might find Peter McWilliams's Do It: Let's Get Off Our Buts interesting and useful, at least in spots. Several section headings are:
2Psychohistorian
Actually, "No valid inference." A man who has admitted in the wrong may or may not be thought much of by Abraham Lincoln is the only thing we can infer, and it's too indefinite to count as a useful deductive truth.

Man, n. An irrational animal whose irrationality is best demonstrated by his irrational belief in his rationality.

-L. A. Rollins, Lucifer's Lexicon

[-]Kyre120

He who knows not and knows not that he knows not is a fool - shun him

He who knows not, and knows that he knows not is a child - teach him

He who knows, and knows not that he knows is asleep - wake him

He who knows, and knows that he knows is wise - follow him

  • Persian proverb
5[anonymous]
I've always enjoyed three-fourths of this quote, but the first line still bothers me. If one knows not that they know not, should we not guide them so they at least know that they know not? Then we can teach them, possibly awake them, and finally have more wise people for others to follow.
1Kyre
Good point. I guess the extra effort and subtlety required to guide the obliviously ignorant makes it tempting to just walk away. Also, the last line presupposes that the knowingly knowledgeable will be wise, which may not be the case if wisdom is taken to have a moral dimension. They could be rational but evil. (Gah - analysing proverbs ...)
7gwern
--Analects 7.8 If you were in China and were confronted with the top 1 billion, would it be worthwhile to try to teach the ignorant who are ignorant even of being ignorant?

Essentially, all models are wrong, but some are useful.

George Box

3gwern
Does this count as a repeat?

I am shamed by my failure. I will master the Search, so that the Search can not master me.

[-]Rain120

Johnny Smith: If you could go back in time to Germany, before Hitler came to power, knowing what you know now, would you kill him?
[...]
Dr. Sam Weizak: I don't like this, John. What are you getting at?
Johnny Smith: What would you do? Would you kill him?
Dr. Sam Weizak: All right. All right. I'll give you an answer. I'm a man of medicine. I'm expected to save lives and ease suffering. I love people. Therefore, I would have no choice but to kill the son of a bitch.

-- The Dead Zone, 1983

[-]khafra140

David: But you're a doctor--you help people!

Dr. Mordin Solus: Lots of ways to help people. Sometimes heal patients; sometimes execute dangerous people. Either way helps.

-- Mass Effect 2

1Christian_Szegedy
I am stunned by the relatively high mod-points of this exchange. I agree that the quotes are moderately funny. (Albeit the M.S. quote was much more funny in the specific context within the game, but even there it was his white-wash response to an action that earned Shepard renegade points.) Still, I can't see, how all this is related to the "art of human rationality"...
6Rain
"Killing is wrong, no matter what," is a very powerful and standard meme for heroes. It is counter intuitive for someone who "loves people" to kill someone. It requires a less-biased assessment of expected utility than is typically performed. That's why I enjoyed the original quote; in the context of the movie, it made sense in the way of typical human failings for him to say no, and his body language and tone highly suggested he would do so right until the end.
6CronoDAS
It's also convenient for writers. Imagine what would happen to the Batman comic book series if someone finally got around to putting a bullet through The Joker's brain. (In a Discworld story, it's suggested that "heroes" and Dark Lords have a bit of an understanding: Dark Lords keep on making all of the mistakes on the Evil Overlord list, and heroes keep on letting Dark Lords escape after the day has been saved.)
3wedrifid
And nauseating. Don't forget nauseating.
1Christian_Szegedy
In my reading, the assessment was funny exactly because it was emotional and therefore biased. That's what use of "son of a bitch" suggested as well.
3Rain
Emotion drives value and purpose; logic is compatible with emotion; Spock is a bad example for rationalists.
3fortyeridania
It's related because it portrays someone disregarding the omission/commission distinction. Among consequentialists (who seem to be quite common on LW), how something happens is not directly relevant to its moral value. Untutored intuition, in contrast, seems to say that killing is worse than letting die. Therefore, if consequentialism is right about this, then many humans' moral intuitions are wrong in a predictable way. Thus they are biased. Thus they are irrational. Thus this is related to the art of human rationality.
8TheOtherDave
The situations in which a habitual killer-of-humans will cause death come up more often than the situations in which a habitual letter-of-humans-die will cause death. If you're a consequentialist who negatively values death, it seems to follow that a habit of killing humans is worse than a habit of letting them die.
1fortyeridania
Thank you for pointing out this argument. I'm not sure I agree that "let-die" situations arise less often than "kill" situations. It seems that every moment you have disposable income (i.e. more than you and yours need to survive) involves a choice between saving someone's life and not saving anyone's life.
0TheOtherDave
(nods) That's fair. And yes, if it makes sense to classify what I'm doing right now as choosing not to avert avoidable deaths, it follows that my lifestyle is morally worse (from a consequentialist perspective) than that of a poor murderer.

An interesting related fact: the British considered assassinating Hitler in Operation Foxley in '44. It was kaiboshed, mostly because he was seen as a really terrible strategist.

New and stirring things are belittled because if they are not belittled, the humiliating question arises, "Why then are you not taking part in them?"

-- H. G. Wells

[-]gwern110

"Perhaps the excellence of aphorisms consists not so much in the expression of some rare or abstruse sentiment, as in the comprehension of some obvious and useful truth in a few words.
We frequently fall into error and folly, not because the true principles of action are not known, but because, for a time, they are not remembered; and he may therefore be justly numbered among the benefactors of mankind who contracts the great rules of life into short sentences, that may be easily impressed on the memory, and taught by frequent recollection to recur habitually to the mind."

--Samuel Johnson, Rambler #175, November 19, 1751

[-]Rain100

Human stupidity is formidable but not invincible.

-- Robert C. W. Ettinger, The Prospect of Immortality

6Vladimir_M
On the other hand... -- Friedrich Schiller Seems to me that Schiller's been better vindicated so far.
2John_Maxwell
Both of them are inaccurate and useless.
3wedrifid
I can not speak for usefulness but "Human stupidity is formidable but not invincible" doesn't seem too off the mark. I suppose it depends on which territory we are considering the stupidity to be formidable in defense of.

...science will save the world. Science is the only thing that can save the world. Science is unstoppable, reason cannot be killed, logic cannot be stopped, there is no force on Earth which can stop a scientist from learning, and turning our backs on science will doom us all. Even the gods are rational and obey laws. The future is not something which happens by just waiting for time to pass. And if you want to be assured of a life after death, you have to build it yourself.

From Fine Structure, by Sam Hughes.

I think that while this quotation is true if we take "SCIENCE!" to mean intelligent optimization pressure, it is far more likely to create affective death spirals around anything that calls itself science than get people to try to fix problems.

2cousin_it
Fine Structure is a great piece of fiction, I started following it around the middle and stayed till the end... but I feel it could've been so much more. Sam started out with some awesome premises, but gradually wrote himself into a corner as the powers in play kept escalating. All the while, his writing skill was noticeably growing stronger and more confident, which is why the series stays readable up to the finale. IMO, the high point is the short story Failure Mode from the middle of the series. It reads like a description of what would happen to Eliezer's Harry Potter if one of his experiments went awry.

The reason Royal Navy [nuclear missles] can be launched without a code is that when it was suggested failsafes should be introduced the British Admiralty took insult at the implication that Officers of the Royal Navy would ever consider launching nuclear missiles without orders or unless it was the correct thing to do.

-- TV Tropes, A Nuclear Error

[-]Cyan70

One day four boys approached Hodja and gave him a bag of walnuts.
"Hodja, we can't divide these walnuts among us evenly. So would you help us, please?"
Hodja asked, "Do you want God's way of distribution or mortal's way?"
"God's way," the children answered.
Hodja opened the bag and gave two handfuls of walnuts to one child, one handful to the other, only two walnuts to the third child and none to the fourth.
"What kind of distribution is this?" the children asked, baffled.
"Well, this is God's way," he answered. ... (read more)

[-]Cyan60

I come in peace. I didn’t bring artillery. But I’m pleading with you, with tears in my eyes: If you fuck with me, I’ll kill you all.

-- Marine General James Mattis, to Iraqi leaders in every area his men served in, after sending his tanks and artillery home following the invasion of Iraq

4NancyLebovitz
Why is this a rationality quote?
2DSimon
My guess is that it's intended as an example of the Prisoner's Dilemma. Then again, nearly every morality-related situation is...
1Cyan
Tit-for-tat in the iterated Prisoner's Dilemma, specifically. It also poses the question: how good a strategy is it to combine humor with mortal threats?
2khafra
It looked to me like a perfect, textbook example of a Schelling-type Threat; but maybe that's just because I'm reading The Strategy of Conflict.
0Cyan
Well spotted; I missed that.

Don't bring a knife to a math fight.

-- "Olyander"

If you don't get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one legged man in an ass kicking contest. You're giving a huge advantage to everybody else.

Charlie Munger

It ought to mean acquiring a method — a method that can be used on any problem that one meets — and not simply piling up a lot of facts.

-- George Orwell

5Theist
I think it would be more clear if it included the previous sentence: Or perhaps just substituting "[Scientific education]" for "It".

Computers are useless. They can only give you answers.

-- Often attributed to Pablo Picasso, but I can't find a reliable source.

I'm quite curious to hear what LW thinks of this one.

9SilasBarta
I don't think it would be controversial to say that "useless" is way too strong a term for describing shortcomings of computers.
2NihilCredo
I can't find the quote's context either, but consider this - why would someone ask Picasso about computers? If the quote is correct, I wouldn't be surprised if it was in response to something like "Do you believe that computers can [be made to] create art [on their own]?". In which case the quote becomes much less categoric.
1SilasBarta
Well, in that case, it still sounds to me like View 2: "Only humans will be able discriminate against art upon learning a computer / monkey / child / prankster made it, pendejo!"
1NihilCredo
But that discussion was about science. Nonhuman science is the same thing as human science, so discriminating is irrational. Nonhuman art is (will) not be (necessarily) the same as human art, and it is quite possible that it will not be at all enjoyable by humans.
2SilasBarta
But it will (likely) be the case that people's opinions about particular artwork will dive sharply downward upon learning it was mostly the work of a computer, even as the pre-revelation opinion is higher than average.
1soreff
And the experimental evidence for this is what? More substantially - it is perfectly possible to have a great deal of difference in the emphasis placed on various subfields in the sciences. If we'd gone directly from vacuum tubes to Drexler/Merkle nanotechnology, do you think semiconductor device physics would have been studied as deeply as it has been?
0fortyeridania
He was no computer scientist, but he presumably knew a lot about specific non-computer things. The more important those things are, the less important computers are likely to be (relative to computers, that is). So I don't think Picasso had to have known much about computers to denigrate them rationally. But I am assuming that Picasso knew some pretty important non-computer things. And it does help to remember when he stopped being able to learn more about computers.
0wedrifid
Particularly if you consider the term literally rather than as a way to say "not particularly important".
4AdeleneDawner
What question was the Avatar movie an answer to? How about the last flash game you played? This conversation?
4WrongBot
It may be worth noting, if the quotation's attribution is accurate, that Picasso died in 1973.
3AdeleneDawner
Okay - assuming that the quote's claim is accurate for its time period, that still leaves the fact that streamlining the process of getting accurate answers leaves more time for figuring out good questions or doing other valuable things.
5WrongBot
I interpreted the quote as being more of a point about answers than about computers. But YMMV.
3Blueberry
I saw it as saying "garbage in, garbage out."
1AdeleneDawner
Peoples' values vary. People who don't value answers wouldn't be very popular if they turned up here, but they do exist, and I don't see much point in passing judgment on them.
6wedrifid
"It's not that I judge them, I just, just..." "Don't see any reason for them to exist?" "Exactly." HP:MoR.
4AdeleneDawner
To be fair, I expect that most of them don't see much reason for people like us to exist, either.
4wedrifid
A practical arrangement all round.
3pjeby
Hey, but I like you! ;-) Seriously, in my line of work, answers only take you a step or two forward. The lasting value is in the questions, which can be reused over and over again to produce change as a side effect of the answering, while the actual answers can be consciously discarded once the process of answering is complete.
2gwern
--'Mentat Zensufi admonition', Chapterhouse Dune; Frank Herbert
1Christian_Szegedy
I think, the quote is useless and in rooted in a fundamental misunderstanding of IT. E.g. experimental mathematics would not exist without computers. Computer simulation is fantastic way to empirically produce and check hypotheses.

"The conformist benefits from the pooled knowledge of it's companions."

-E.O. Wilson in Sociobiology the new synthesis

[-][anonymous]40