Seems like a useful tool for a cynical observer is: "When you see a person X saying/doing something to a person Y, think about whether it would make sense to describe this interaction as a signal being sent from X to some third party Z. (Even in cases when there is no such person Z present. Sometimes people signals to themselves.)"
For example, my actions may be inefficient to stop you from breaking social norms, but that's okay, because the purpose is to signal to outsiders that I am not the kind of person who breaks social norms.
[Some things are better done when you don’t think about them. This has an interesting extension when you consider situations where the default response does very little to actually solve things. Maybe we’ve been Goodharted by social incentives.]
Cue the four stages of competence.
The four stages represent a framework for modeling how people get better at a task, from unconscious to conscious incompetence, followed by conscious to unconscious competence.
In other words:
What gets touted as the highest stage of mastery here is that of unconscious competence, when you get so good at a task that you don’t need to focus on the task at hand to do well at it.
Some quick examples that come to mind:
One common thread to observe about the above three examples is that they all involve a “performance” of sorts. Especially in the cases of physical activities, it’s easy to chalk it up to “muscle memory” and consider the case of unconscious competence closed. What is interesting for me right now, though, is the paradoxical nature of the generalized concept where thinking about how to do X well is antithetical to doing X well.
A few more examples:
There’s been a lot of talk recently on LessWrong about fake frameworks and how they relate to what’s true and/or correct. I think the above three examples do a good job of showcasing the value of framings other than the directly-optimizing/overanalyzing mindset rationalists are often stereotypically depicted as having.
And if you really were trying to do well, then the right thing to do, under a direct optimizing mindset, would be to self-modify such that you weren’t thinking that much about how well you were doing. (In order to do well.)
The big picture takeaway here is to just be mindful of how your internal state can have instrumental effects on what you do—including the states that are typically induced when you’re trying to do something.
Another interesting phenomenon that seems like a similar instance of this property is when you generalize it to the ways people respond to problems. In particular, certain systems seem to be set up in such a way that selection pressures mean that the people you want to affect are the ones least likely to respond to whatever intervention you’ve set up.
Once again, to triangulate this concept, here are three examples:
What seems striking here to me is how quickly these default responses spring to mind in each of the systems. See a sad friend? You tell them that “it’ll be okay”. Feel outraged by the other side? An outburst jumps straight to your lips.
These responses are downright ineffective. They don’t work, and we should know from doing them time and time again, that they don’t work. So why do these responses persist?
It’s things like this which make me sympathetic to a Hansonian view of human behavior. In these situations, I don’t think we’re taking a good look at the results of our actions. Instead, it’s like we’re just going through the motions to send a message about the roles we play:
Going off our above examples:
I feel like this is one the areas in life where we’ve Goodharted ourselves. It’s not as if people don’t care about the results of the actions they take. People do care about what happens to them [citation needed]. Rather, somewhere along the line, we’ve convinced ourselves that the effective action is the one which just so happens to do a very good job of serving a social function but quite a sucky job of doing the actual thing.