# 99

Over the past few years, we have discreetly approached colleagues faced with a choice between job offers, and asked them to estimate the probability that they will choose one job over another. The average confidence in the predicted choice was a modest 66%, but only 1 of the 24 respondents chose the option to which he or she initially assigned a lower probability, yielding an overall accuracy rate of 96%.

—Dale Griffin and Amos Tversky1

When I first read the words above—on August 1st, 2003, at around 3 o’clock in the afternoon—it changed the way I thought. I realized that once I could guess what my answer would be—once I could assign a higher probability to deciding one way than other—then I had, in all probability, already decided. We change our minds less often than we think. And most of the time we become able to guess what our answer will be within half a second of hearing the question.

How swiftly that unnoticed moment passes, when we can’t yet guess what our answer will be; the tiny window of opportunity for intelligence to act. In questions of choice, as in questions of fact.

The principle of the bottom line is that only the actual causes of your beliefs determine your effectiveness as a rationalist. Once your belief is fixed, no amount of argument will alter the truth-value; once your decision is fixed, no amount of argument will alter the consequences.

You might think that you could arrive at a belief, or a decision, by non-rational means, and then try to justify it, and if you found you couldn’t justify it, reject it.

But we change our minds less often—much less often—than we think.

I’m sure that you can think of at least one occasion in your life when you’ve changed your mind. We all can. How about all the occasions in your life when you didn’t change your mind? Are they as available, in your heuristic estimate of your competence?

Between hindsight bias, fake causality, positive bias, anchoring/priming, et cetera, et cetera, and above all the dreaded confirmation bias, once an idea gets into your head, it’s probably going to stay there.

1Dale Griffin and Amos Tversky, “The Weighing of Evidence and the Determinants of Confidence,” Cognitive Psychology 24, no. 3 (1992): 411–435.

## New to LessWrong?

New Comment
Some comments are truncated due to high volume. Change truncation settings

I hate changing my mind based on my parents' advice because I want to demonstrate that I'm capable of making good decisions on my own, especially since we seem to disagree on some fundamental values. Specifically, they love their jobs and put a moral value on productivity, while my goal in life is to "work" as little as possible and have as much "fun" as possible.

4[anonymous]11y
Eliezer never said to change your mind based on wrong advice! However, if you feel as if you should be following your parents' advice, perhaps you should question exactly how capable you really are (at the moment).

Does this mean that if we cannot remember ever changing our minds, our minds are very good at removing clutter?

Or, consider a question that you've not made up your mind on: Does this mean that you're most likely to never make up your mind?

And, anyway, in light of those earlier posts concerning how well people estimate numeric probabilities, should it be any wonder that 66% = 96%?

7DanielLC14y
Don't they normally make them more certain? Like, if they're 96% sure, there's a 66% chance that they're right, rather than the other way around?

Not to argue, but to point out, that this is not necessarily a bad thing. It depends entirely on the basis of one's conclusion. Gut instincts are quite often correct about things we have no conscious evidence for - because our unconscious does have pretty good evidence filters. Which is one of the reasons I suggested rationalization is not necessarily a bad thing, as it can be used to construct a possible rational basis for conceptualizations developed without conscious thought, thus permitting us to judge the merit of those ideas.

Here is one way to change your mind. Think through something carefully, relying on strong connections. You may at some point walk right into a conclusion that contradicts a previous opinion. At this point something will give. The strength of this method is that it is strengthened by the very attachment to your ideas that it undermines. The more stubborn you are, the harder you push against your own stubbornness.

I agree with Adirian that not changing our minds is not necessarily a bad thing.

The problem, I guess, like with most things is we can't be sure which way to go. Gut feelings are often quite correct. But how do we know when we are having a bias which is not good for us and when it's a gut feeling? Gut feelings inherently aren't questionable. Biases need to be kept in check.

If we run through the standard biases and logical fallacies like a checklist and what we think doesn't fall in any of them, we can go with our gut instinct. Else, give whatever we have in...

It probably doesn't help to live in a society where changing one's positions in response to evidence is considered "waffling", and is considered to show a lack of conviction.

Divorce is a lot more common than 4%, so people do admit mistakes when given enough evidence.

1Viliam_Bur12y
Changing your mind or "updating" is not necessarily a sign of rationality. You could also update for wrong reasons. For example, a divorce can happen when a person has unrealistic expectations on marriage. Updating their beliefs about their partner would be just a side effect of refusing to update their beliefs about marriage. Also, in some cases, the divorce could have been planned since the beginning (for example for financial gain), so it actually did not include a change of mind.
5Peterdjones12y
I think the embargo on mind-changing is a special case for politiicians: after all, if they say one thing on the hustings, and then do another in office, that makes a mockery of democracy. However, if it is applied to non-pliticians, that would be fallacious.
4sparkles11y
If they say one thing and intend to do another, sure - but if they actually update? That may be bad PR, but I don't think it's undemocratic.
-6Peterdjones11y
0Izeinwinter11y
You misrepresent democracy very badly in the above post. Politicians are not agents of the voters, they are representatives of them, appointed by, and accountable to the demos, but not a mirror of it- they are not supposed to enact the policies voters thought appropriate 2 years ago at the polls, or what polls well today. They are supposed to do what the voters would want done if they had time to research the issue and give it some thought, incorporating all data about the present situation. If policy was supposed to reflect the averaged will of the people politicians would be entirely redundant and we could just do lawmaking by popular initiative.
0Peterdjones11y
Of course it is unworkable for politicians to stick rigidly to their manifestos. It is also unworkable for them to discard their manifestos on day one.
1tlhonmey3y
On the other hand, of the people I know who have gotten divorced, refusal to admit mistakes seems to be one of the leading causes...

I wonder if the act of answering the question actually causes the decision to firm up. Kind of the OvercomingBias Uncertainty Principle.

It is nice to have a clear example of where people are consistently underconfident. Are there others? Michael, good point about divorce.

-8Peacewise12y

I second Robin's question.

I'd also like to learn whether the experimental finding holds for a wide variety of decisions. (Eliezer mentioned only picking a job offer.)

Aren't people consistently underconfident when it comes to their money? Everybody does something, invest in something, but aren't really sure about it even after they've done it. It's in its most extreme when it comes to the stock market.

Another instance is when people approach members of the opposite sex who they think are attractive. They consistently misunderestimate themselves.

Otherwise it depends on what their used to, like people in technology are underconfident when it comes to negotiation and so forth.

In the case of Divorce, the reasons cannot always be taken as evidence for the marriage having been a mistake to begin with.

Things happen and people change.

This is an interesting idea and doesn't surprise me given thin-slicing behavior and the like. But the research itself seems a little thin. Where is the actual testing versus a control group? What about other decisions that don't involve jobs?

Also, I think probably we know what we will choose 99% of the time because we make the decision instantaneously. The real question is whether we do this even on decisions that we don't consciously know what we are going to choose. Are we as accurate in those decisions?

It is nice to have a clear example of where people are consistently underconfident. Are there others?

People tend to take into account the magnitude of evidence (how extreme is the value?) while ignoring its reliability, and they also tend to be bad at combining multiple pieces of evidence. So another good way to generate underconfidence is to give people lots of small pieces of reliable evidence. (I believe it's in the same paper, "The Weighing of Evidence and the Determinants of Confidence".)

I recall having an argument over dinner with a friendly acquaintance about an unimportant but interesting problem. I thought about it for few days and decided he was right. I've hated him ever since.

4DanielLC14y
And now we're curious. What was the problem?

Are you they as available, in your heuristic estimate of your competence?

I'm unable to parse this sentence.

3bigjeff513y
Drop the "you" and see the linked "Availability Heuristic".

I used to have a button that said "If you haven't changed your mind lately, how do you know you've still got one?" I really liked that sentiment.

It's very easy to get comfortable with our opinions and beliefs, and uncomfortable about any challenge to them. As I've posted elsewhere, we often identify our "selves" with our "beliefs", as if they "were" us. Once we can separate our idea of "self" as different from "that which our self currently believes", it becomes easier to entertain other thoughts...

A lot of people probably already know that, it's a familiar "deep wisdom", but anyway: you can use this not-changing of your mind to help you with seemingly complicated decisions that you ponder over for days. Simply assign the possible answers and flip a coin (or roll a dice, if you need more than 2). It doesn't matter what the result is, but depending on wether it matches your already-made decision you will either immediately reject the coin's "answer" or not. That tells you what your first decision was, unclouded by any attempts to j...

0DaFranker12y
Without knowing the terms or technical explanation for it, this is what I have always been doing automatically for as long as I can remember making decisions conciously (generously apply confidence margin and overconfidence moderation proportional to applicable biases). However, upon reading the sequences here, I realize that several problems I have identified in my thought strategies actually stem from my reliance on training my intuition and subconscious for what I now know to be simply better cached thoughts. It turns out that no matter how well you organize and train your Caches and other automatic thinking, belief-forming and decision-making processes, some structural human biases are virtually impossible to eliminate by strictly relying on this method. What's more, by having relied on this for so long, I find myself having even more difficulty training my mind to think better.

That's true. Matters are not helped by the value society places on commitment and consistency. When we do, in fact, change our minds, we are more often than not labeled as "wishy-washy," or some similarly derogatory term.

This article reminds me of the movie "Inception"... once an idea is planted it is hard to get it out.

As Eliezer says, on short time scales (days, weeks, months) we change our minds less often than we expect to. However, it's worth noting that, on larger time scales (years, decades) the opposite seems to be true. Also, our emotional state changes more frequently than we expect it to, even on short time scales. I can't seem to recall my exact source on this second point at the moment (I think it was some video we watched in my high school psychology class), though, anecdotally, I've observed it to be true in my own life. Like, when I'm feeling good, I m...

I would say that the study by Griffin and Tversky is incomplete. The way I see it, we have an inner "scale" of the validity of evidence and decide based on that. As was pointed out in one of the previous posts, we should bet on an event 100% of the time if the event is more likely than the alternatives. Something similar is happening here, where if we are more than 50% sure that job A is better than job B, we should pick job A. Given that the participants were 66% sure, this would mean that there is a low a priori probability for them to change their minds...

[-]Flow2mo10

The principle of the bottom line

I think "The Bottom Line" here is meant to link to the essay.