TLDR: First you go with your gut, then you get a logical model, then you improve that model. Trusting your logical model over your gut before it gets good enough is a very common way to believe wrong things. [epistemic status: probably approximately true, with possible pathological cases around the...
Rationality is believing true things and making good choices. This phrasing isn't quite as precise as it could be, but it communicates the gist of both epistemic and instrumental rationality in very simple language. I use this wording when communicating what rationality is to non-rationalists, and it seems to do...
TLDR: First you go with your gut, then you get a logical model, then you improve that model. Trusting your logical model over your gut before it gets good enough is a very common way to believe wrong things.
[epistemic status: probably approximately true, with possible pathological cases around the edges]
The process of getting better at describing and predicting things seems to usually go something like this:
First, you start out with an intuitive model which nature gives you automatically without any effort on your part. This model uses the language of system 1, and is a black box whose contents are unknown to you.
Then, you develop a weak analytical model in the language... (read 582 more words →)
Both "believing true things and making good choices" and "believing false things and making bad choices" constrain anticipation. For example, someone doing the former more often is more likely to do well on a math test than someone doing the latter more often. They are both meaningful. The thing that they aren't is controversial. One feels like an obviously good idea and the other feels like an obviously bad idea. (note: I said "feels like an obviously good idea" instead of "obviously is a good idea" because of the whole sacrifice element Qiaochu_Yuan brought up.)
Oh, interesting. To want to leave alone people who aren't willing to make the right kinds of sacrifices makes a lot of sense to me. I can come up with objections off the top of my head, and objections to those and so on, so I haven't thought about it enough to agree or disagree with the idea. The reason I was aiming for agreeableness is that the phrase is intended for answering questions like, "What are your hobbies?" or "What is your friend so-and-so like?" in a short way that communicates the basic idea and keeps the conversation pleasant. I honestly hadn't considered protecting people from rationality. But now that it's... (read more)
Sacrifice is definitely an important part of the process, yes. But I think waxing poetic about how miserable it is to be rational to an average person will make them think we are very weird and likely not correct, while my phrase is really difficult to disagree with and so is less likely to invite criticism. Both my phrase and talking about sacrifice are honest, but the former has a better effect on my status. I think it is a better PR move, basically, to have an agreeable elevator pitch.
Rationality is believing true things and making good choices.
This phrasing isn't quite as precise as it could be, but it communicates the gist of both epistemic and instrumental rationality in very simple language. I use this wording when communicating what rationality is to non-rationalists, and it seems to do its job well enough. Maybe other rationalists will find it useful too.