Note: this post originally appeared in a context without comments on Overcoming Bias. Old comments on this post are over here.
How should we deal with the cases when epistemic rationality contradicts instrumental? For example, we may want to use placebo effect because one of our values is that healthy is better than sick, and less pain is better than more pain. But placebo effect is based on the fact that we believe pill to be a working medicine that is wrong. Is there any way to satisfy both epistemic and instrumental rationality?
It depends from case to case, I would think. There are instances when you're most probably benefited by trading off epistemic rationality for instrumental, but in cases where it's too chaotic to get a good estimate and the tradeoff seems close to equal, I would personally err on the side of epistemic rationality. Brains are complicated, forcing a placebo effect might have ripple effects across your psyche like an increased tendency to shut down that voice in your head that talks when you know your belief is wrong on some level (very speculative example), for limited short-term gain.
It seems to me that this is not a contradiction of two rationalities. Rather, it is similar to the resonance of doubt. If a placebo works when you believe in it, that means that if you believe in it, it will be true. Here you need a reverse example, when if you believe that something is true, then it becomes false. (Believing that something is safe again won't work, since you just need to not act more carelessly based on the safety of something, which is just a matter of instrumental rationality)
If you believe that the placebo works, it works. You're right in believing it works.
If you don't believe that the placebo works, it doesn't work. You're right believing it doesn't work
If you believe that the sky is blue, you're right.
If you believe that the sky is green, it's still blue, you're wrong.
Truths that have humans involve some amounts of reflexivity.
I'd say you shouldn't force yourself to believe something (epistemic rationality) to achieve a goal (instrumental rationality). This is because, in my view, human minds are addicted to feeling consistent, so it'd be very difficult (i.e., resource expensive) to believe a drug works when you know it doesn't.
What does it even mean to believe something is true when you know it's false? I don't know. Whatever it means, it'd have to be a psychological thing rather than an epistemological one. My personal recommendation is to only believe things that are true. This is because the modern environment we live in generally benefits rational behavior based on knowledge anyway, so the problem doesn't need to surface.
The essay reminds me of the book 𝑳𝒂𝒏𝒈𝒖𝒂𝒈𝒆 𝒐𝒏 𝑻𝒉𝒐𝒖𝒈𝒉𝒕 𝒂𝒏𝒅 𝑨𝒄𝒕𝒊𝒐𝒏 by Samuel Hayakawa. The author also used the map and territory metaphor in the book.
Thank you for the reference. I just stumbled into this website and found the essays interesting to me. As a Chinese reader there is not so many this kind of contents in chinese web. Really lucky to enjoy the thought while improving my English.
Looking back on this post having learned about the Free Energy Principle, I wonder if instead of saying "rationality is improving accuracy of beliefs and achieving your values", one could instead just say "rationality is the minimization of free energy". The free energy principle, or more specifically the theory of active inference, encodes both goals of "having good models" and "achieving your values" as simply minimizing the difference between observation and prediction, either by making the prediction more accurate through more comprehensive models, or by finding the best policy (action) upon the world that most likely to produce observations more in line with an existing model.
You cannot change reality, or prove the thought, by manipulating which meanings go with which words.
The same word can mean many things, words that have convergent evolution in their sounding but different meanings are spelled differently for a reason. Propaganda is manipulating the meaning of things, this is often done with slogans and words. Lies are the changed meaning of things to shape reality. Reality is a perception from a particular perspective as in the anthropic problems, it is relational not necessarily objective.
Creating a definition can be done, and is at times useful to make sense of and verify the likeness of maps and territory contained in other people's heads. Such to confirm the maps of language and words are congruent.
If things can not be defined the definition is left up to the individual and open to interpretation. The utility of this experiential approach allows individuals to engender their own ideas. When reading around a philosophical work and engaging with the material you build a representation of its meaning. As you do every time you read or write a word. Even where philosophical works have definitions there is often further assumed knowledge to decode and grasp the work in its entirety. In both cases where there is a formal definition, examples and implementation of its usage, this adds meaning and information.
Where the probability of controversy high and the ability to quell controversy is low, the probability of formal defence of ideas is reduced. There is a ceiling but unto time to which, things can be defended, defined or explained.
We need not provide and defend formal definitions, a definition is defined through usage. If the probability of a definition causing controversy is high and defining it has low utility the importance of a formal definition is decreased. Leaving things in ambiguity or with multiple degrees of interpretation limits reprisals.
If you don't have anything nice to say don't allow it to take shape, to become definitive. This is besides the point that communication can still transmit useful information.
The fact that there is no definition is the definition and is evidence for the definition. You can define things, but in the experiential sense what can you do with information that is wrong to steel-man it, to give it utility and make it useful.
If the benefit of a definition providing epistemic accuracy is lower than the instrumental utility of not defining, why define it?
Ultimately if we are to become rational the worst way to brainstorm is to have an anchoring effect around a definition of rationality that also causes controversy. As in the Stability–instability paradox, not naming something creates more names not of the thing in actuality but ideas around it. We are the Blind men yet but touching the elephant that is rationality.
you should substitute more specific language in place of “rational”: “The self-benefiting thing to do is to run away, but I hope I would at least try to drag the child off the railroad tracks,”
Wouldn't it be correct to say that it would be 'instrumentally rational' to run away in this case? It sounds rational to me, as far as you 'winning' means you 'surviving'.
I think by winning, he meant: "art of choosing actions that lead to outcomes ranked higher in your preferences", though I don't completely agree with this word choice of "winning" which could be ambiguous/causing confusion.
A bit unrelated, but more of a general comment on this - in my belief, I think people generally have unconscious preferences, and knowing/acknowledging these before weighing out preferences are very important, even if some preferences are short term.
Is the last sentence rational?
The one that says "If you speak overmuch of the Way, you will not attain it."
This is a reference to Taoism (the tao = the Way). I believe it is a different approach to the tenet I've heard expressed as "The Tao that can be explained is not the true Tao". I believe the reference is meant to remind us that the point here is to end up performing less wrong rational thinking, not just talking about it.
Nice discussion. Thanks for putting this together. I learned something about Epistemic rationality vs Instrumental rationality.
The bit about the sky being blue or green seems to beg the question of a justification for objective truth as championed by St Augustine and Leibniz as opposed to arguments for subjective reality as championed by the Cynics and Skeptics and, more recently, the Frankfort school. One could make the case that the sky appears green to one but blue to another.
This topic comes up in many places throughout the history of thought. I'm actually working on a post for my blog exploring that at www.SimplyUrban.Org.
We mean:
If that seems like a perfectly good definition, you can stop reading here; otherwise continue.
Sometimes experimental psychologists uncover human reasoning that seems very strange - for example, someone rates the probability "Bill plays jazz" as less than the probability "Bill is an accountant who plays jazz". This seems like an odd judgment, since any particular jazz-playing accountant is obviously a jazz player. But to what higher vantage point do we appeal in saying that the judgment is wrong?
Experimental psychologists use two gold standards: probability theory, and decision theory. Since it is a universal law of probability theory that P(A) ≥ P(A & B), the judgment P("Bill plays jazz") < P("Bill plays jazz" & "Bill is accountant") is labeled incorrect.
To keep it technical, you would say that this probability judgment is non-Bayesian. Beliefs that conform to a coherent probability distribution, and decisions that maximize the probabilistic expectation of a coherent utility function, are called "Bayesian".
This does not quite exhaust the problem of what is meant in practice by "rationality", for two major reasons:
First, the Bayesian formalisms in their full form are computationally intractable on most real-world problems. No one can actually calculate and obey the math, any more than you can predict the stock market by calculating the movements of quarks.
This is why we have a whole site called "Less Wrong", rather than simply stating the formal axioms and being done. There's a whole further art to finding the truth and accomplishing value from inside a human mind: we have to learn our own flaws, overcome our biases, prevent ourselves from self-deceiving, get ourselves into good emotional shape to confront the truth and do what needs doing, etcetera etcetera and so on.
Second, sometimes the meaning of the math itself is called into question. The exact rules of probability theory are called into question by e.g. anthropic problems in which the number of observers is uncertain. The exact rules of decision theory are called into question by e.g. Newcomblike problems in which other agents may predict your decision before it happens.
In cases like these, it is futile to try to settle the problem by coming up with some new definition of the word "rational", and saying, "Therefore my preferred answer, by definition, is what is meant by the word 'rational'." This simply begs the question of why anyone should pay attention to your definition. We aren't interested in probability theory because it is the holy word handed down from Laplace. We're interested in Bayesian-style belief-updating (with Occam priors) because we expect that this style of thinking gets us systematically closer to, you know, accuracy, the map that reflects the territory. (More on the futility of arguing "by definition" here and here.)
And then there are questions of "How to think" that seem not quite answered by either probability theory or decision theory - like the question of how to feel about the truth once we have it. Here again, trying to define "rationality" a particular way doesn't support an answer, merely presume it.
From the Twelve Virtues of Rationality:
We are not here to argue the meaning of a word, not even if that word is "rationality". The point of attaching sequences of letters to particular concepts is to let two people communicate - to help transport thoughts from one mind to another. You cannot change reality, or prove the thought, by manipulating which meanings go with which words.
So if you understand what concept we are generally getting at with this word "rationality", and with the sub-terms "epistemic rationality" and "instrumental rationality", we have communicated: we have accomplished everything there is to accomplish by talking about how to define "rationality". What's left to discuss is not what meaning to attach to the syllables "ra-tio-na-li-ty"; what's left to discuss is what is a good way to think.
With that said, you should be aware that many of us will regard as controversial - at the very least - any construal of "rationality" that makes it non-normative:
For example, if you say, "The rational belief is X, but the true belief is Y" then you are probably using the word "rational" in a way that means something other than what most of us have in mind. (E.g. some of us expect "rationality" to be consistent under reflection - "rationally" looking at the evidence, and "rationally" considering how your mind processes the evidence, shouldn't lead to two different conclusions.) Similarly, if you find yourself saying "The rational thing to do is X, but the right thing to do is Y" then you are almost certainly using one of the words "rational" or "right" in a way that a huge chunk of readers won't agree with.
In this case - or in any other case where controversy threatens - you should substitute more specific language: "The self-benefiting thing to do is to run away, but I hope I would at least try to drag the girl off the railroad tracks" or "Causal decision theory as usually formulated says you should two-box on Newcomb's Problem, but I'd rather have a million dollars."
"X is rational!" is usually just a more strident way of saying "I think X is true" or "I think X is good". So why have an additional word for "rational" as well as "true" and "good"? Because we want to talk about systematic methods for obtaining truth and winning.
The word "rational" has potential pitfalls, but there are plenty of non-borderline cases where "rational" works fine to communicate what one is getting at, likewise "irrational". In these cases we're not afraid to use it.
Yet one should also be careful not to overuse that word. One receives no points merely for pronouncing it loudly. If you speak overmuch of the Way you will not attain it.