Previously in seriesSelecting Rationalist Groups
Followup toRationality is Systematized Winning, Extenuating Circumstances

Why emphasize the connection between rationality and winning?  Well... that is what decision theory is for.  But also to place a Go stone to block becoming a whining-based community.

Let's be fair to Ayn Rand:  There were legitimate messages in Atlas Shrugged that many readers had never heard before, and this lent the book a part of its compelling power over them.  The message that it's all right to excel—that it's okay to be, not just good, but better than others—of this the Competitive Conspiracy would approve.

But this is only part of Rand's message, and the other part is the poison pill, a deadlier appeal:  It's those looters who don't approve of excellence who are keeping you down.  Surely you would be rich and famous and high-status like you deserve if not for them, those unappreciative bastards and their conspiracy of mediocrity.

If you consider the reasonableness-based conception of rationality rather than the winning-based conception of rationality—well, you can easily imagine some community of people congratulating themselves on how reasonable they were, while blaming the surrounding unreasonable society for keeping them down.  Wrapping themselves up in their own bitterness for reality refusing to comply with the greatness they thought they should have.

But this is not how decision theory works—the "rational" strategy adapts to the other players' strategies, it does not depend on the other players being rational.  If a rational agent believes the other players are irrational then it takes that expectation into account in maximizing expected utility.  Van Vogt got this one right: his rationalist protagonists are formidable from accepting reality swiftly and adapting to it swiftly, without reluctance or attachment.

Self-handicapping (hat-tip Yvain) is when people who have been made aware of their own incompetence or probable future failure, deliberately impose handicaps on themselves—on the standard model, in order to give themselves an excuse for failure.  To make sure they had an excuse, subjects reduced preparation times for athletic events, studied less, exerted less effort, gave opponents an advantage, lowered their own expectations, even took a drug they had been told was performance-inhibiting...

So you can see how much people value having an excuse—how much they'll pay to make sure they have something outside themselves to blame, in case of failure.  And this is a need which many belief systems fill—they provide an excuse.

It's the government's fault, that taxes you and suppresses the economy—if it weren't for that, you would be a great entrepreneur.  It's the fault of those less competent who envy your excellence and slander you—if not for that, the whole world would pilgrimage to admire you.  It's racism, or sexism, that keeps you down—if it weren't for that, you would have gotten so much further in life with the same effort.  Your rival Bob got the promotion by bootlicking.  Those you call sinners may be much wealthier than you, but that's because God set up the system to reward the good deeds of the wicked in this world and punish them for their sins in the next, vice versa for the virtuous:  "A boor cannot know, nor can a fool understand this: when the wicked bloom like grass and all the doers of iniquity blossom—it is to destroy them till eternity."

And maybe it's all true.  The government does impose taxes and barriers to new businesses.  There is racism and sexism.  Scientists don't run out and embrace new ideas without huge amounts of work to evangelize them.  Loyalty is a huge factor in promotions and flattery does signify loyalty.  I can't back religions on that divine plan thing, but still, those wealthier than you may have gotten there by means more vile than you care to use...

And so what?  In other countries there are those with far greater obstacles and less opportunity than you.  There are those born with Down's Syndrome.  There's not a one of us in this world, even the luckiest, whose path is entirely straight and without obstacles.  In this unfair world, the test of your existence is how well you do in this unfair world.

I earlier suggested that we view our parents and environment and genes as having determined which person makes a decision—plucking you out of Platonic person-space to agonize in front of the burning orphanage, rather than someone else—but you determine what that particular person decides.  If, counterfactually, your genes or environment had been different, then it would not so much change your decision as determine that someone else would make that decision.

In the same sense, I would suggest that a baby with your genes, born into a universe entirely fair, would by now be such a different person that as to be nowhere close to "you", your point in Platonic person-space.  You are defined by the particular unfair challenges that you face; and the test of your existence is how well you do with them.

And in that unfair challenge, the art of rationality (if you can find it) is there to help you deal with the horrible unfair challenge and by golly win anyway, not to provide fellow bitter losers to hang out with.  Even if the government does tax you and people do slander you and racists do discriminate against you and others smarm their way to success while you keep your ethics... still, this whole business of rationality is there to help you win anyway, if you can find the art you need.  Find the art together, win together, if we can.  And if we can't win, it means we weren't such good rationalists as we thought, and ought to try something different the next time around.  (If it's one of those challenges where you get more than one try.)

From within that project—what good does a sense of violated entitlement do?  At all?  Ever?  What good does it do to tell ourselves that we did everything right and deserved better, and that someone or something else is to blame?  Is that the key thing we need to change, to do better next time?

Immediate adaptation to the realities of the situation!  Followed by winning!

That is how I would cast down the gauntlet, just to make really, really sure we don't go down the utterly, completely, pointlessly unhelpful, surprisingly common path of mutual bitterness and consolation.


Part of the sequence The Craft and the Community

Next post: "Mandatory Secret Identities"

Previous post: "Incremental Progress and the Valley"

New Comment
99 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

One of the greatest benefits I've gotten from (westernized) Buddhism is the idea that a resistance to reality is at the root of much unhappiness.

It seems absurd to me that the human mind so constantly wishes that reality was different - I don't see how it serves our evolutionary needs. But while I don't have an explanation, it is amazing how often I find myself denying reality instead of "Immediate adaptation to the realities of the situation! Followed by winning!". For example, when I encounter bad, unexpected auto traffic, whining is such a horribly unproductive reaction that it still boggles my mind every time I do it. Yet in many moods (already tired, stressed) it is my default response.

I think many rationalists would get a lot more personal happiness out of working on this single concept, as well as improving strategy for our causes, than many of the narrower and more complex ideas presented on OB/LW.

I heartily endorse this sentiment, but it's important to remember that wishing reality was different can also be valuable motivator. I'm pretty good at accepting the things I cannot change. The problem is that this sometimes generalizes too broadly, and leads me to accept things that I probably shouldn't. My emotional reactions don't always have the wisdom to know the difference.
I'm talking about present reality, not future reality. Our mind doesn't seem to distinguish very well, as you mention w/ your emotional reactions. We imagine that the current context is different in the same way that we imagine the future could be different, even though the current situation is (tautologically) unchangeable. The question of whether future reality can be changed is far from straightforward, but at least there's a shot. Whereas we know for sure that the inputs we are experiencing in the present moment cannot be changed. We can act in the next moment to change things in the moment after that, but nothing will change the fact that I've encountered an unexpected traffic jam.
"I don't see how it serves our evolutionary needs." Ever heard of the concept of group selection? Evolution does not just happen at the level of individual genes, it can also take place at the societal level. If we accept the axiom that human beings have been living in ethnic and tribal units since the beginning of our species' history-- a valid assumption considering that we evolved on the African savannah and had to compete with both apex predators and powerful herbivores-- then society will select for those traits that it deems most suitable for its continued survival, and groupthink appears to be one of them. Am I implying that social conformity is the cause of most of our problems? There's actually quite a bit more truth to that statement than most people are willing to acknowledge, but the point of this blog post is that you can still make it in spite of all the barriers society puts up. I for one do not disagree.

Well, I do have quite a bit of the "bitter loser" in me, but I don't go blaming other people for my failures. All I do is waste my time reading blogs on the internet and playing video games, so, as that other guy named Buffett put it, it's my own damn fault.

If he's so smart, why isn't he rich?
Note: Jimmy Buffett, the singer-songwriter who wrote the song "Margaritaville", is, indeed, rich. Just not as rich as that other Buffett.

"But this is only part of Rand's message, and the other part is the poison pill, a deadlier appeal: It's those looters who don't approve of excellence who are keeping you down."

As lethal as I'm sure it will be to speak even faint praise of a person that is so widely hated that expressing loathing of her is a common 'applause light'...

That's not what Rand's message was. It wasn't even part of her message. One of her main points was, to use the classical phrase, that evil is ultimately impotent. The power of evil to harm comes entirely from the failure of good to recognize it and refuse to loan it its own power.

Rand's message was that people were keeping themselves down, that they had bought into ethical and ideological positions and accepted them without questioning, that they had accepted teachings which passed off poorly-disguised wolves as lambs long before they'd developed the critical thinking skills to evaluate the teachings. And that the teachings were that white was black and black was white, etc.

I am often struck that the people who declaim Rand's writings and ideas most vehemently, especially those that use their proclaimed disapproval to win the approval of others, almost always hold up crude parodies of what Rand actually said in the process, and rarely address her actual positions and their strengths and weakness (of which there are many in both categories).

"One of her main points was, to use the classical phrase, that evil is ultimately impotent. The power of evil to harm comes entirely from the failure of good to recognize it and refuse to loan it its own power."

This is a defense of Rand? I agree it's one of her main points. Also completely false, to the point where I consider it a classic error of people trying to reform social systems. The idea that if you can just expose the evil of the system, that will fix the problems.

Intuitive, noble, and totally wrong when applied to a world where evil most often emerges from the behavior systems which are not easily understood or modified.

No one said that exposing evil would be sufficient, and heroes of Rand's novels didn't just go around exposing evil. It helps, though.
Where do you think social systems get their power? People give it to them, then the population becomes ignorant and/or apathetic and allows it to run amok with no attempts to strip it of the power they gave it. The idea isn't that you just expose the evil. You have to deny it power over you.
If each individual denies it power, it will have no power. If half of them give it power, it will have a lot of power, whether or not you're one of the people giving power. You're just one person. You don't give it much power. But if you learn the system, and figure out how to meddle in it, you could weaken it, make it lean more towards doing good, or harness it for your own gain.

I do think Rand was being a bit more complex than that. The whole point of "Atlas" is: the heroes are failing to win because they insist on acting as though they were in an ideal fair world, but those who who accept the status quo and work to win inside it will end up burned worse, because the system is structured to corrupt and consume them - meanwhile our heroes escape with virtue intact. "Atlas" constructs a spread of parasitic, beaten, adapting, fair-but-accepting, and fair-and-renouncing characters to illustrate this. Rand is trying to say "a rationalist who understood the rules of the game would decide not to play".

Really, the fault with "Atlas" is that it posits an awful world-spanning System that in factual reality, just doesn't exist. And without that premise it's two inches of wasted paper.

"Really, the fault with "Atlas" is that it posits an awful world-spanning System that in factual reality, just doesn't exist." I can't agree with that. I don't believe there's some secret, scheming Conspiracy making schools stunt the intellectual development of children. Nevertheless, that is the overwhelmingly common outcome in my society. There's no System trying to corrupt the world. Just lots of individual actors acting in accordance with that they perceive their interests to be. There's no Invisible Hand, either. Yet markets self-organize.
Oh come on, "it could be self-organizing" is so obvious I left it out. Your entire comment was wasted unless you can show there actually is some self organizing evil.
What Rand says is more like "An awesome rationalist who understood the sick twisted rules of the game would leave and start their own game and not stick with those awful losers who make the world suck." So sure, Atlas explicitly encourages embracing the reality of an unfair world full of parasites - the heroes' character progression comes through that acceptance. But the characters of Atlas implicitly encourage whining and bitterness, which are symptoms of failing to accept the reality of an unfair world. And I think the implicit message affects readers much more strongly.

I'm glad to see that many others have pointed out EJ's mistaken interpretation of Objectivism. To add a prototypical passage to demonstrate the error:

Ellsworth Toohey: There's the building that should have been yours. There are buildings going up all over the city which are great chances refused and given to incompetent fools. You're walking the streets while they're doing the work that you love but cannot obtain. This city is closed to you. It is I who have done it! Don't you want to know my motive?

Howard Roark: No.

Ellsworth Toohey: I'm fighting you an

... (read more)

I always interpreted the 'Looters and Moochers' differently; a corollary to the 'It's okay to Win,' statement saying 'It's okay that others Lose - they did so by their own hand.' Rather than offering an excuse for Rationalists/Ubermenschs/Super-Geeks to say 'Nice guys finish last,' I read it as an indictment of that very behaviour. Only 'Looters and Moochers' make excuses, blame others, and fault circumstances - the Super-Geek Wins despite all of those.

I'd wager that Ayn Rand would agree with me if I said this to her (if she wasn't too busy denouncing me ... (read more)

Did they say "I'm a winner" in your presence? How did you know what they imagined? That you felt a "creepy vibe" says more about you than them. Where else have you felt this "creepy vibe"? I don't see a lot of extensional facts in your criticisms of Objectivists. I just see that you clearly don't like them. The Objectivists I have personally known have been fine, decent, fun people. A married couple that went off to be professors at the University of Georgia. I knew the husband better, and played tennis with him while we were both in grad school. Neither of us were very good, but it was exercise. They invited me over to their house a few times to play bridge and have drinks with some of their other friends, all Objectivist leaning, if not Objectivists. I always had a good time. They never told me they were "Winners". The discussions were lively, honest, and interesting. They gave me quite a pleasant vibe, of honest, rational people who didn't have a lot of time for trying to getting ahead by snearing at other people.
Even if the winners cheated?
When it comes to the real world rather than games, claiming that there is such a thing as cheating is a form of self-handicapping.
There's no actual cheating?I guess we'd better free Bernie Madoff then.

Doing things other people don't like isn't cheating, but punishing people for doing things you don't like isn't cheating either, and doing things that other people don't like without taking the possible punishment into account isn't rational (= leads to not-winning).

This is an atypical definition of "cheating".
Hm. *checks a dictionary* The dictionary says you're right; apparently the standard definition of 'cheating' is that it refers to deceptive behavior. I'd been using it to refer to not abiding by agreed-upon rules, either explicitly or in spirit (e.g. munchkinism). I think this is a more accurate definition, given that there are some games where deception is an expected part of the game, and deception is not considered cheating in those cases (e.g. Diplomacy). In the real world, there are no agreed-upon rules to break (I never agreed not to murder anyone...), so 'cheating' doesn't apply.
Well if I say: "I will build a hot-air balloon" then it's reasonable to interpret that as agreeing to the rule "I have to build a hot-air balloon", so if I don't, I'm cheating. And then it's reasonable to extend that to other kinds of statements, like "I built a hot-air balloon" Speech in Diplomacy, it seems, is not quite real speech. The default position is that speech is true.
That doesn't seem reasonable to me, actually. I interpret it as 'I intend to build a hot-air balloon', which is much weaker evidence about future world-states even if it's true. (It's also stronger evidence about current world-states.) This strikes me as naive. In my experience, most people don't lie without a reason to do so, but also most people will lie when they do have such a reason, and such reasons are fairly common. Our society is built on that assumption, in some ways, even - it's practically required that one make up an excuse to leave a conversation with an annoying person rather than tell them that you don't want to talk to them, for example.
It doesn't, because rule contravention is not the sole sufficient condition of cheating. Cheating involved 1) breaking rules that 2) others are following for 3) advantage whilst 4) disguising the fact.
Anyone who is convicted in a court of law has failed to abide by agreed-on rules
The point was that the convicted person did not agree to the rules. That some other people agreed on them is irrelevant to Adelene's point. Also, not all people convicted in a court of law actually did the thing they were convicted of.
Yes, exactly this. It seems reasonable to me to describe laws as "rules that the government acts as if all citizens have agreed to abide by", at least for values of 'acts as if' that apply to the judicial system. The government acting that way results in a system that works reasonably well as far as I can tell, and the fact that the government acts that way makes it generally reasonable to act as if one has agreed to follow those rules. But that's not the same as actually agreeing to follow those rules, and the most rational way of handling the situation is to keep that in mind and actually do a cost/benefit analysis when something illegal seems like it might be worthwhile anyway - and, such a cost/benefit analysis should take all the results of the action into account, including e.g. the possibility of the laws being changed to restrict further behavior of that type, or the possibility of getting a problematically bad reputation, or more subtle issues.
I don't think Bernie Madoff was making a principled protest against the inquities of the financial regulators: he was quite happy for other people to abide the rules. (Reliant on that: if everyone cheats, cheaters have no edge). I chose him as an example, rather than, eg Mandela for a reason. I know. I hoped I could take all the side-conditions about fair trials etc as read.
In a perfectly acceptable sense of the word "cheating" Madoff cheated people out of their money.
What sense is that?
Telling lies for profit. Financial fraud.
The question isn't whether we should free Bernie Madoff; it's whether Ayn Rand would do so (if even that).

disclaimer: the ranty part is not directed at yudkowsky

"From within that project - what good does a sense of violated entitlement do? At all? Ever? What good does it do to tell ourselves that we did everything right and deserved better, and that someone or something else is to blame? Is that the key thing we need to change, to do better next time?"

I dunno. I don't follow that many competitive endeavours but the people who cast about looking for excuses after a loss tend to be pretty good. Admittedly the people who go on about what a bitch you... (read more)

I think American Atheists might be better than objectivists as an example of a whining-based rationalist community.

I'm not actually convinced that either are particularly rational, as a rule, except insofar as both have built communities around ideas that are mostly correct.

I come hailing as a more learned Objectivist than I was before. This article actually caused me to go find an online Objectivist community for the purpose of observing them to see if your assertion was true. I've found that it is not. I have not met a single "whiny" Objectivist out of all of the Objectivists I now chat almost-daily with.

Objectivism holds a primacy of existence attitude towards reality, as opposed to a primacy of consciousness attitude. This means that reality comes before our wishes, and if we want our wishes to come true, we hav... (read more)

For the purposes of this discussion, I don't think that's true. Most criticisms of Rand can be effectively rebutted by showing their inconsistency with her fiction. No real need to get into her essays or The Introduction to Objectivist Epistemology.
You can't criticise North Korea until you've read all of Kim Il Sung in the original Korean.
The more information from all sources you base your criticism on, the more you can rely on it being substantive and accurate.

I think it's perfectly possible to maximize your outcome given current conditions while still being resentful that it is only a local maxima and there are much higher hills that you are being prevented from climbing.

I have two objections: Being resentful of inevitable reality (I'm not tall enough or fast enough to ever make it onto the NBA, no matter how hard I practice) makes about as much sense as being angry at the sky for being blue, or at your eyes for only having three colour sensors. Yes, it sucks, but reality isn't an entity which you can influence by yelling at. This sort of resentment is counter-productive. In the second sense, bitterness can become an excuse for why you never cross over to the ideal Maxima. Are you sure this is the maximum of your potential? Really? Really? The only thing you can control in this world is yourself; assigning agency to outside forces detracts that agency from your own abilities. Even if you're right, and fate has it in for you, growing resentful will do nothing but make things worse. Your maximum while being resentful is not as high as your maximum without it.
you're speaking a little more concretely. I was more thinking "we don't have flying cars because NASA and the FAA suck".

You know, I hadn't noticed before, but the claim that rationality should make you win is isomorphic to a similar contention I have about ethics. I guess that shouldn't be surprising since ethics is tied to decision-making, and so is this definition of 'rationality'.

So I'll come out and say it - ethics specifies criteria for judging which (character|actions|outcomes) are the best.

Rationality tells you how to achieve your goal. Ethics tells you your goal.

"And if we can't win, it means we weren't such good rationalists as we thought, and ought to try something different the next time around."

This attitude, that somehow, every single obstacle to success or happiness is solved by rationality, is a mistake, I think. People are not in control of the amount of opportunity they have, and i don't think being supremely rational is a sure way to triumph. Victims of slavery and car crashes are extreme examples, but I think there's more subtle situations in which no reasoned plan of action can straightforwardly help you "win."

1Said Achmiz
The point, I think, is not that you can achieve arbitrarily large success no matter what your starting point. It's that you achieve the most success that is possible given your starting point. In other words: yes, life is unfair. You now have two options. You can either: 1. Do the best you can with what you are given. This doesn't mean quietly acquiescing to whatever default fate is expected of you, or going the path of least resistance; it can mean doing grandiose, ambitious, seemingly-crazy things. Maybe you decide that your best option, given your starting point, is to try to change the world (or at least some relevant part of it). But at any rate, your strategy takes your starting point as given. or... 2. Sit there and whine that the world isn't fair, while behaving as if the world actually did work the way you think it should work. This strategy will, of course, fail. It seems obvious to me that whatever external factors have conspired to keep you down, at least part of the responsibility for such failure is your own. ETA: I think one source of contention in your comment is that when we talk about "triumphing", or "solving" an obstacle, what we mean is simply achieving the best result given the starting conditions, rather than achieving some given point on an absolute scale of success.

This raises the question of what positive attributes we can attempt to apply to this little sub-culture of aspiring rationalists. Shared goals? Collaborative action?

Some have already been implying heavily that rationality implies certain actions in the situation most of us find ourselves in, does it make sense to move forward with that?

Is success here just enabling the growth of strong rationalist individuals, who go forth and succeed in whatever they choose to do, or to shape a community, valuing rationality, which accomplishes things?

I think my problem is that far too often I make decisions as if I am in the "should universe" described here:

One of the failure modes I've come to better understand in myself since observing it in others, is what I call, "living in the should-universe". The universe where everything works the way it common-sensically ought to, as opposed to the actual is-universe we live in. There's more than one way to live in the should-universe, and outright delusional optimism is only the least subtle. Treating the should-universe as your point

... (read more)

It's those looters who don't approve of excellence who are keeping you down. Surely you would be rich and famous and high-status like you deserve if not for them, those unappreciative bastards and their conspiracy of mediocrity.

Any Objectivists who believe this have missed half of Ayn Rand's message and are doing Objectivism completely wrong.

Not only did they miss one of the main points of John Galt's three hour long speech in Atlas Shrugged, but people who level this accusation against Objectivism as a whole missed it as well.

The point I'm referring t... (read more)

I agree that the "it's not my fault, it's everyone else keeping me down" sentiment is entirely antithetical to Objectivism. Indeed, one of the clearest distinctions between the good guys and bad guys in Atlas Shrugged is that the good guys are focused on getting things done, no matter what, regardless of whatever obstacles are thrown in their path by the villains, while the bad guys are always making excuses and looking to blame others. However, I think it probably is correct to say that many individual members of the Objectivist movement did exhibit this kind of behavior, at least some of the time. Sadly, Rand in her later life and many of her closest followers were often decidedly poor exemplars of their purported ideas, and it's valid to criticize Rand as such. But that doesn't necessarily mean that the philosophy itself is naturally prone to this vice.

Let's be fair to Ayn Rand

Well, let's. Other than secondary characters like The Fountainhead's Henry Cameron (a great architect whose spirit has been broken), which of Rand's heroes are like this?

Surely you would be rich and famous and high-status like you deserve if not for them, those unappreciative bastards and their conspiracy of mediocrity.

4Eliezer Yudkowsky
A fair question. But an old, bitter, un-achieving character isn't likely to inspire much heroic empathy and desire to emulate. So Rand didn't write John Galt that way; she wrote the villains that way. The question is what happens when some real-world person takes John Galt as a role model. In the story he's not just a former great physicist, he's all buff and heroic and has his own little survivalist ranch and society actually falls down without him. But in reality...
So the answer to Arundelo's question would be "none of Rand's heroes are that way, while her villains are that way." Wouldn't such a choice by a writer generally indicate disapproval of such a trait? Particularly given Rand's theory of Romatic Art, I'd say that's a certainty in her case. I have a rather limited sample of Objectivists that I have known sufficiently well to know that they in fact took Rand seriously. 3 people. They all turned out quite well. What's your data? I can't detect an actual case being made in your comments, though I think I see a lot of innuendo. Do you think you've made a clear and compelling case? Could you spell it out for me if you think you did?
The first example to come to mind is Richard Halley from Atlas Shrugged, but I don't remember the book all that well.

You're showing stellar advertising skills with this WIN thing. I'm nowhere close. Honest applause and upvote. Now we gotta prepare something for the inevitable moment when the masses come and ask us how to WIN.

"There's not a one of us in this world, even the luckiest, whose path is entirely straight and without obstacles. In this unfair world, the test of your existence is how well you do in this unfair world."

Of course shutting up and multiplying, always advancing forward and doing the impossible is the way to go. But as said, the test of instrumental rationality is whether or not you succeed at what you've set yourself up to.

If rationality is being signaled for intellectual honesty's sake alone, or maybe pride, then this signal may not correlate w... (read more)

Another great post, thanks Eliezer! But, if rationality is for you to win, shouldn't you try to keep it a secret from others? Like if you knew a way to make money in the stock market would you spread it if that nullified your advantage?


Winning isn't necessarily zero-sum.

Two things: * Advantage over others is not the only thing people care about. * The "rationality" developed in secret is unlikely to grow more powerful than whatever technology a single farmer from Dark Ages could develop in a lifetime, that is to say not impressive at all.
Regarding the second point: that's why Guilds were created, and they were quite powerful in their day. Why do you think they're called 'trade secrets'?
But modern technological civilisisation didn't take of until the guild system (keep it secret) was replaced by the patent system (publish it)
From Newcomb's Problem and Regret of Rationality (with emphasis added): So yes, if for you winning means making money, and your best strategy to do that is to take advantage of irrationality in the stock market, then you will be motivated to keep your methods of rationality secret. If, on the other hand, your utility function has a term for others, then you will want to teach them to be rational and win.
Eliezer is trying to win by creating a Friendly AI. If he gets more people to help him, this will help him win. If he spreads rationality, this will get more people to help him. Thus, he is spreading rationality to help him win.

Is it possible that humans, with their limited simulation abilities, do not have the mental computational resources to simulate an irrational persons more effective beliefs?

This would mean that the 'irrational' course of action would be the more effective.

Even if they can't model their behaviors like they do for normal people, that doesn't mean there is some systematic way of rationally predicting their behaviors.

Immediate [and optimal] adaptation to the realities of the situation! Followed by winning!

The first sentence is rationality as process; the second is rationality as outcome [winning].

Shouldn't a "rationality is ..." slogan communicate both aspects, and not just one or the other?

I realize that "systematized winning" sort of hightlights both aspects, but I think that it still seems to imply that it's primarily about winning, when it's about both equally.

"Systematized winning" shouldn't be about both equally, in an "A, and also B" sort of sense. It should be about a particular relationship between process and outcome: about using thinking techniques that help one win, whatever those turn out to be. "Rationality is using whatever thinking techniques actually help you build useful world-models", or some pithier re-wording. Then, when you combine that aim with the empirical claim that in fact to attain unusual success in a domain it helps to follow particular processes (e.g., to accept that the domain works the way it works; to gather evidence as to what practices are actually most likely to help you succeed rather than defending your first hypothesis against all objections; etc.), you end up with "systematized winning" having some implications about process. But it'd be nice if the slogan captured that the processes "rationality" might advocate are a means to the end of accurate beliefs and/or winning (and that the specific notion of how "rational people" think should be changed, if it turns out that our processes don't help with accurate beliefs and/or winning), and that process goals have zero rationality-goodness in themselves, apart from their consequences.
It should be about using thinking techniques that help one win, whatever those turn out to be. Absolutely. And rationality is the means by which we evaluate, implement, and update those techniques. The "means by which" in the previous sentence is what rationality is. But the process stuff is here a means to an end (to be changed, if it turns out that our processes don't help), rather than a reified end in itself. You're talking here about specific processes for achieving specific wins. I'm saying that rationality integrally involves the higher-level general processes by which we determine which techniques (low-level processes) to use, how to evaluate the results of using them, how to modify them based on experience, etc.

I see your point and your purpose, but I have 2 caveats:

  • The fact that it would be wonderful and inspirational if rationality was always the winning strategy, doesn't mean that rationality is always the winning strategy.
  • I think there are a lot of losers who handicap themselves by making excuses; and there are a lot of winners who believe that winning proves virtue, and not winning proves a lack of virtue. Both are wrong. But the winners are in the positions of power; and so their errors do more damage.

The whole issue resists analysis, because "m... (read more)

The fact that it would be wonderful and inspirational if rationality was always the winning strategy, doesn't mean that rationality is always the winning strategy.

I would offer that rationality is not a winning strategy, it is a meta-strategy for identifying winning strategies.

6Eliezer Yudkowsky
I would agree but put "rationality" in quote marks, that is, it is the subject of the discipline named "rationality" to find rational strategies.
Did you mean "to find winning strategies", or are you using those synonymously? Either way, I agree with the reference/value distinction here.
5Eliezer Yudkowsky
I guess I was trying to make a side point by using the two as though they were synonymous. Maybe the precise way would be that "instrumental rationality" is the study of systematically winning strategies, just like "epistemic rationality" is the study of systematically accurate guessing.
You have just passed the recursive buck. Identifying winning strategies and then using them is also a winning strategy, an adaptive one, which may make it stronger. It is this strength that matters, not the ritual.
9Eliezer Yudkowsky
That's a virtuous pass, not a vicious pass. Deriving instrumental utility on the reflective level from instrumental utility on the object level is just what we want. Defining truth on the object level by invoking a definition of truth on the meta-level would be vicious.
I would go even farther than you: rationality is an infinitely-recursive process of evaluation whose fundamental principle is consistency.
3Eliezer Yudkowsky
I am interested in how this infinite recursion manages to complete in finite time.
0Paul Crowley
Would you prefer "unboundedly recursive"? So on any given occasion it will only recurse to a finite depth, but there's no bound on the depth of its recursion?
1Eliezer Yudkowsky
That's fine, but I've never seen a "pass the recursive buck" scenario that actually did work by requiring only a finite recursive depth on any given occasion. It always degenerates into an infinite hierarchy of ordinals that you can't describe without creating a new hierarchy on top. Well, I mean yes there are programming exercises for computing the Fibonacci numbers; I'm referring to when this trick is tried in epistemology or logic.
3Paul Crowley
I have in mind a scenario something like what Dennett describes in Consciousness Explained: we imagine that our awareness of our own thoughts is in some mysterious way infinitely recursive, because when we go looking for a bound on how many times we can repeat the step of becoming aware of the previous level of awareness, we don't find one; but the bound arrives exactly whenever we care to stop looking. There's no bound on how often we can reflect on the way we're deciding a particular question and decide if that, in turn, is rational, but there will have to come a point at which you have to stop recursing if you want to actually decide the base question. There again, it may be a mistake to look for a sensible meaning in Annoyance's usual vague crap.
2Paul Crowley
I find this one of the hardest and most enlightening disciplines of being a materialist. It certainly is the single thing that puts the most distance between me and most people in the way I think about the world. I keep hoping to either write or read a top-level post about this.
6Eliezer Yudkowsky
Causality and Moral Responsibility (It's being reduced to parts, not explained away.)
The issue isn't whether or not someone who whines is factually correct. It's whether or not whining is a winning strategy. It generally isn't, so don't wine.

I don't have anything to say but...

fuck yeah!

2Eliezer Yudkowsky
I hope this is being voted down because people don't like profanity and not because this is a public expression of assent in a rationalist community.
I would guess a combination of unnecessary profanity and lack of informational content. Censuring "Me too!" posts is a long-standing and entrenched norm of internet culture. At any rate, I expect that a similarly phrased post expressing disagreement would likely be voted down as well, possibly even more strongly.