Note, though, that if you fail to achieve a correct answer, it is futile to protest that you acted with propriety. "Rationalists should win" is still normative, even if "Rationalists always win" isn't true.
Another thing that's pretty crucial here is that rationality is only aimed at expected winning.
Suppose we live on Lottery Planet, where nearly everyone has a miserable life, but you can buy a lottery ticket for a chance of $BIGNUM dollars. Nonetheless, the chances of winning the lottery are so small that the expected value of buying a ticket is negative. So the rational recommendation is to refrain from buying lottery tickets.
Nonetheless, the agents who would be "smiling down from their huge piles of utility" could only be the ones who "irrationally" bought lottery tickets. (Credit for this example goes to someone else, but I can't remember who...)
You shouldn't expect rationality to help you win absolutely. Some people will just get lucky. You should expect it to help you do better than average, however. The rationalist on lottery planet is certainly likely to be doing better than the average lottery-ticket buyer.
On a similar note: rationally succeeding and simply succeeding might involve two entirely different approaches. For example, if success is largely a result of other successful people conferring success on you because they see you displaying certain signals, it doesn't follow that gaming the system will be as easy as naturally producing those signals. Signalling often relies on displays that are difficult to fake. The cognitive resources needed to fake it are often vastly disproportionate to the resources used in sincere signalling and, regardless, in many cases we may not even know what the signals are or how to fake them. The rational road to, say, political success might involve a multibillion dollar research program in social neuroscience whereas the natural road simply involves being born into the right family, going to the right schools, etc, and naturally acquiring all the signalling associated with that.
I am wary of excluding work ethic and mastery over akrasia from rationality, and I am not sure about intelligence.
Akrasia and work ethic are choices. Aspiring rationalists who find themselves not making the choices they have found to be rational should seek to remedy this situation, not excuse themselves for having akrasia.
Some limitations on how rational you can be might be unfair, but that doesn't stop them from making you irrational.
The problem with this is that multiple motivation systems contribute to action, and only one of them looks anything like "do the thing I expect will achieve my goals given what I believe about the world." For example, I wouldn't call a blind reflex a "choice" or "decision."
I'm having trouble calling a person who can rattle out a perfectly rational thing to do in every circumstance but instead spending their life complaining about how they would do this and that if only they didn't have akrasia a rational one.
With that terminology, I would read shminux's comment as saying: "I have trouble calling rational a person who is an expert on rationality but not an expert at rationality." Where is the failure?
There's a tale of Naive Agent. When the Naive Agent comes across a string, NA parses it into a hypothesis, and adds this hypothesis into his decision system if the hypothesis is new to NA (it is computationally bounded and doesn't have full list of possible hypotheses); NA tries his best to set the prior for the hypothesis and to adjust the prior in the most rational manner. Then the NA is acting on his beliefs, consistently and rationally. One could say that NA is quite rational.
Exercise for the reader: point out how you can get NAs to give you money by c...
rationality is a normative concept defined by logic, Bayesian probability theory, and Bayesian decision theory
Compare with what Russell said about mathematics:
"Pure Mathematics is the class of all propositions of the form “p implies q,” where p and q are propositions containing one or more variables, the same in the two propositions, and neither p nor q contains any constants except logical constants."
Where human rationality is concerned, simple measures analogous to deductive correctness in mathematics don't capture many important aspect...
It would be more correct to say that "Winning as defined by general society norms" is not a strong measure of rationality. "Actual Winning", as defined by the agent's own total values, certainly must be.
Also, while the prior probability of winning is (should be) higher in the rationality group, and lower outside, there are likely still many more winners outside the rationality group, because there are so many more people outside it than within. Making use of the availability heuristic to estimate "winning" and decide whether rationality pays off won't work well.
Rational thinking has helped me overcome my akrasia in the past, so if someone isn't very good at overcoming theirs, I see that as weak evidence of poor rationality.
Thus, one could have highly rational beliefs and make highly rational choices and still fail to win due to akrasia, lack of resources, lack of intelligence, and so on. Like intelligence and money, rationality is only a ceteris paribus predictor of success.
I disagree here. Akrasia, resources, and intelligence are all factors that should be taken into account by a rational agent. The reason rational agents don't always win is the complexity of factors in the real world is too great to predict reliably, no matter how rational and intelligent you are. Rationality provides the best possible means of "balancing the odds", but nothing can guarantee success.
First, it seems to me that this is mainly a debate over the definition of instrumental rationality. And I suspect the reason people want to have this debate is so they can figure out whether they count as especially "instrumentally rational" or not.
The simplest definition of "instrumentally rational" I can think of is "a person is instrumentally rational to the extent they are good at acting to achieve their goals". Thus somebody with akrasia would not qualify as a very instrumentally rational under this simple definition. You...
It seems to me that the LessWrong rationality does not concern itself with the computational limitations of the agents, using as normative the idealized model that ignores limitations, and lacks extensive discussion of comparative computational complexities of different methods, as well as the security of the agent from deliberate (or semi accidental) subversion by other agents. (See my post about naive agent)
Thus the default hypothesis should be that the teachings of LessWrong for the most part do not increase the efficacy (win-ness) of computationally bo...
Someone who claims to have read "the vast majority" of the Sequences recently misinterpreted me to be saying that I "accept 'life success' as an important metric for rationality." This may be a common confusion among LessWrongers due to statements like "rationality is systematized winning" and "be careful… any time you find yourself defining the [rationalist] as someone other than the agent who is currently smiling from on top of a giant heap of utility."
So, let me explain why Actual Winning isn't a strong measure of rationality.
In cognitive science, the "Standard Picture" (Stein 1996) of rationality is that rationality is a normative concept defined by logic, Bayesian probability theory, and Bayesian decision theory (aka "rational choice theory"). (Also see the standard textbooks on judgment and decision-making, e.g. Thinking and Deciding and Rational Choice in an Uncertain World.) Oaksford & Chater (2012) explain:
Is it meaningful to attempt to develop a general theory of rationality at all? We might tentatively suggest that it is a prima facie sign of irrationality to believe in alien abduction, or to will a sports team to win in order to increase their chance of victory. But these views or actions might be entirely rational, given suitably nonstandard background beliefs about other alien activity and the general efficacy of psychic powers. Irrationality may, though, be ascribed if there is a clash between a particular belief or behavior and such background assumptions. Thus, a thorough-going physicalist may, perhaps, be accused of irrationality if she simultaneously believes in psychic powers. A theory of rationality cannot, therefore, be viewed as clarifying either what people should believe or how people should act—but it can determine whether beliefs and behaviors are compatible. Similarly, a theory of rational choice cannot determine whether it is rational to smoke or to exercise daily; but it might clarify whether a particular choice is compatible with other beliefs and choices.
From this viewpoint, normative theories can be viewed as clarifying conditions of consistency… Logic can be viewed as studying the notion of consistency over beliefs. Probability… studies consistency over degrees of belief. Rational choice theory studies the consistency of beliefs and values with choices.
Thus, one could have highly rational beliefs and make highly rational choices and still fail to win due to akrasia, lack of resources, lack of intelligence, and so on. Like intelligence and money, rationality is only a ceteris paribus predictor of success.
So while it's empirically true (Stanovich 2010) that rationality is a predictor of life success, it's a weak one. (At least, it's a weak predictor of success at the levels of human rationality we are capable of training today.) If you want to more reliably achieve life success, I recommend inheriting a billion dollars or, failing that, being born+raised to have an excellent work ethic and low akrasia.
The reason you should "be careful… any time you find yourself defining the [rationalist] as someone other than the agent who is currently smiling from on top of a giant heap of utility" is because you should "never end up envying someone else's mere choices." You are still allowed to envy their resources, intelligence, work ethic, mastery over akrasia, and other predictors of success.