I like Scott’s post on what LessWrong has learned in its lifetime. In general I approve of looking back at your past misunderstandings and errors, and trying to figure out what you did wrong. This is often very hard, because it’s hard to remember or imagine what absurd thoughts (or absences of thought) could have produced your past misunderstandings. I think this is especially because nonsensical confusions and oversights tend to be less well-formed, and thus less organizable or memorable than e.g. coherent statements are.

In the spirit of understanding past errors, here is a list of errors which I think spring from a common meta-error. Some are mentioned in Scott’s post, some were mine, some are others’ (especially those who are a combination of smart and naive I think), a few are hypothetical:

• Because I believe agent-like behavior is obviously better than randomish reactions, I assume I am an agent (debunked!).
• Because I think it is good to be sad about the third world, and not good to be sad about not having enough vitamin B, I assume the former is why I am sad.
• Because I explicitly feel that racism is bad, I am presumably not racist.
• Because my mind contains a line of reasoning that suggests I should not update much against my own capabilities because I am female, presumably I do no such thing.
• Because I have formulated this argument that it is optimal for me to think about X-risks, I assume I am motivated to (also debunked on LW).
• Because I follow and endorse arguments against moral realism, and infer that on reflection I prefer to be a consequentialist, I assume I don’t have any strong moral feelings about incest.
• Because I have received sufficient evidence that I should believe Y, I presumably believe Y now.
• I don’t believe Y, and the only reason I endorse to not believe things is that you haven’t got enough evidence for them, therefore I must not have enough evidence to believe Y.
• Because I don’t understand the social role of Christmas, I presume I don’t enjoy it (note that this is a terrible failing of the outside view: none of those people merrily opening their presents understands the social role either).
• Because I don’t endorse the social role of drinking, I assume I don’t enjoy it.
• Because signaling sounds bad to me, I assume I don’t do it, or at least not as much as others.
• Because I know the cost of standing up is small (it must be, it’s so brief and painless!), this cannot be a substantial obstacle to going for a run (debunked!).
• I know good motives are better than bad motives, so presumably I’m motivated by good motives (unlike the bad people, who are presumably confused over whether good things are the things you should choose)
• I have determined that polyamory is a good idea and babies are a bad idea, therefore I don’t expect to feel jealousy or any inclination to procreate, in my relationships.

In general, I think the meta problem is failing to distinguish between endorsing a mental characteristic and having that characteristic. Not erroneously believing that the two are closely related, but actually just failing to notice there are two things that might not be the same.

It seems harder to make the same kind of errors with non-mental characteristics. Somehow it’s more obvious to people that saying you shouldn’t smoke is not the same as not smoking.

With mental characteristics however, you don’t know how your brain works much at all, and it’s not obvious what your beliefs and feelings are exactly. And your brain does produce explicit endorsements, so perhaps it is easy to identify those with the mental characteristics that the endorsements are closely related to. Note that explicitly recognizing this meta-error is different from it being integrated into your understanding.