I'm really surprised by the clear logic of this. Lots of interesting corollaries follow:
Roles are an efficient optimization to use instead of acting dumb (provided you got trained in the right ones).
To train your chosen roles is a good idea.
Roles can be tuned to better match your needs.
It should be possible to look for missing roles covering interesting behavior patterns.
All of the above implies rational behavior can be trained.
The clear exposition makes this Main material although I wonder whether that applies to cross-posted material. I also wonder how to place this in the sequences context. Seems to be related to Teachable Rationality Skills and Rationality Dojo. It also reminds me of Geek Fu - which I did consider as rather humorous until now.
EDIT: Typos fixed.
You know, I think Peter Quill used Geek Fu at the end of Guardians of the Galaxy: he distracts the seemingly indestructible bad guy at a key moment by doing something so bizarre that the bad guy stops to stare at him trying to figure out what the hell is going on.
Good post; you should put it in Main. It's nice to see more stuff here on some of the aspects of instrumental rationality from HPMoR. Similarly, the two linked posts on your blog are brilliant; I'd been vaguely intending to write about them myself at some point, but you might have saved me the work.
Roles allow real-time action under stress.
There is a trade-off, of course: your reaction is fixed by training and it's not necessarily appropriate to the specific situation.
Another point is that I am not sure you're speaking of the same thing when you talk about the training of instinctual response in martial arts and about taking on particular roles. Taking on a role is basically an exercise in self-perception, it is sometimes expressed in terms of wearing a particular hat. It's not that you need to react so fast that you conscious mind doesn't have time to get involved, it's more like you pre-select a viewpoint from which your conscious mind will evaluate the situation and choose what to do.
While true most of the time this doesn't always seem to apply. Every now and then I find myself in a discussion with my close friends defending some viewpoint which doesn't quite make as much sense as I would have liked it to make, and the times I noticed this in time and tried to figure out why I was taking those viewpoints I noticed that they were the natural viewpoints for my role at that moment, which would explain a lot.
In other words: switching roles can be hard, sometimes while 'playing a role' the conscious mind need not be involved too much. This is, of course, precisely the reason to adapt roles.
Yes. On thing that happens a lot for me is that I fall into the role of Defense Counsel - the role, not the profession, i.e. defending those absent or otherwise unable to defend themselves. Say somebody attacks a person or viewpoint. It's quite quite likely that I will fall into the role of the defender of that person or viewpoint - even though I don't agree with that viewpoint at all!
You might want to work on using role that "notices when argues for a side instead of evaluating for which side to argue". From rationality habits this might be one relatively simple to implement. Of course I have to work on it as well
I now notice when I do argue for the absent side and make this clear. Before I just assumed that other people would take arguments as elucidations of facts as I do - and then it doesn't matter who takes whose 'side'.
This isn't a very concrete comment, I'll just point out some connections I'm seeing with a few dual-process theories of mind. For instance, a la Robin Hanson, the actions that we take / signals that we give in near mode are especially telling to those around us, and his theory of identity is that we want to give off a set of reliable signals (so as to be trustworthy), and so it makes sense that there are various coherent personalities that we can run. Also, Joshua Greene makes the case for a dual-system theory of ethical decision making, that our inflexible yet highly efficient System 1 comes up with intuitions that let us run our every day lives, and our considered, yet highly inefficient System 2 comes up with our reflective, utilitarian judgements, and can allow us to overrule our intuitions sometimes. Your identity/role, in near mode, seems to tell your System 1 what to be thinking, and your System 2 can change what identity/role you're running at a given time.
I used to love Rock Band. One day when playing a particularly difficult guitar solo on expert I nailed 100%… except “I” didn’t do it at all. My eyes saw the notes, my hands executed them, and no where was I involved in the process. It was both exhilarating and creepy, and I basically dropped the game soon after.
One man's modus ponens...
See, I used to notice this in my guitar playing.
Now I'm a classically-trained concert musician.
I find it quite inteesting to watch my fingers do things that I don't feel I'm controlling. It's interesting if I notice I'm making a mistake, I can even shift my hand position to allow a better reach, without making a conscious decision to do so.
One man's modus ponens...
One man's "creepy" is another man's "self-improvement". :D
Works for rationality, transhumanism, pickup arts, and even guitar playing.
I also had that same experience on the higher levels of Rock Band. I am not talented with any real-life musical instruments, but you say you feel that with guitar; for you personally, is that an episodic thing, or does that consistently happen when playing serious guitar? Is that something that most musicians know about, because it was exquisitely bizarre--is that the secret allure of musicians? Or does one build up a tolerance that drives one toward excellence in the hopes of catching the "high of accomplishment"?
Generally, it slips under your radar; it's not really relevant, it doesn't change anything. I think noticing it is just as a result of a) being very reflective and b) being in a music school where every practice room has a mirror in. Your encouraged to observe yourself play, to see it from other angles. It's just like realising you're walking somewhere without really exerting any conscious effort, except you're doing something more specialised. No, I don't think it's generally why people become musicians. That's more to do with the music itself, normally. And I didn't quite understand your last sentence.
I was kind of going off on a speculative tangent on that last sentence. I was wondering if that feeling was somehow reward-system related, and would fuel a musician's drive to excel. Like they try to play better and better to achieve that euphoria which only comes on when they do better than they ever have, with diminishing (dopamine?) returns, but, as a side-effect, increasing their practical talent to ever higher levels. So the musical prodigy over time becomes motivated more by the tangible rewards (fame, increased income), which will never compare to the feelings that made him choose that path in the first place. It would apply to many careers if it was a valid theory.
Oh no. I didn't mean to imply anything that... Romanticised. Certainly for me, the returns from being able to play the guitar have increased as I've been able to play better.
Recently I started to notice this kind of third-person perception of situations where I explain things that I understand well enough. The perception is more vivid when the topic is unusual, things I might've thought about a lot but only rarely discussed in similar contexts. I'm guessing explanation of a familiar topic can run mostly on System 1, freeing System 2 to observe the performance, and unfamiliar circumstances make the performance more interesting, hence perception of it more vivid.
A great article, Eneasz.
Reminds me of something that is sitting in my quotes file, apparently coming from a Navy SEAL:
"Under pressure you don't rise to the occassion, you sink to the level of your training."
Ensure that you have trained enough for the next challenge, because it is the training that will see you through it, not your agenty conscious thinking.
Spent the last 5+ years of my life trying to do this, specifically for the role of Nurse (and Lifeguard before that). It's been fairly successful, and even generalizes a little–I am frequently the Person who Gets Shit Done in non-nursing contexts.
I'm not sure that the competence/learned skill routines/martial arts for rationality aspect is the same thing as "Being Responsible For This Shit." The former is something that takes years of doing hard things over and over, training the right mental motion the same way you'd train the physical motion. Almost all the things that actually make me a competent ICU nurse fall into this category.
The latter is something that can change in a day, with the right mental reframe. (Example: I usually basically never volunteer to drive places, although I've learned how–I'm not super comfortable and I don't have to. Then I was The Person In Charge of logistics for a large event, and hardly anyone else could drive, and I was responsible–so rather than spend a ton of energy convincing other people to drive places for me, I just got in the car.)
The two skills are probably related and probably correlated–for example, I suspect that many people have trouble taking on the role of "Person In Charge" because they have low confidence in their ability to actually take the right action and make things better rather than worse. (Given that in plenty of situations, taking the wrong action confidently is worse than doing nothing, that may be justified). Acquiring competence in one area, like nursing, brings confidence, and I think that's the thing that generalizes to the rest of my life, rather than any of the specific routines and skills and dealing-with-emergency templates that I've spent years training. It feels like I have a good understanding of which situations actually require very little skill, where the main thing is having the necessary confidence to speak. (Then again, I'm not sure I could distinguish this from "having ingrained a skill to the point that it doesn't even feel like a skill anymore.)
This correlates with my experience in the military. I had a job for a while that did not allow time for thoughtful analysis before each decision. In order to become competent, I had to do simulation after simulation after simulation, then live exercise after live exercise after live exercise...to the point where I could just react (hopefully competently).
Although I was well-trained in that role, it didn't automatically make me good at "Being Responsible For This Shit". Being Responsible (well, being good at Being Responsible) requires consideration of additional factors above and beyond your own skill-set when making decisions. I couldn't have been In Charge without having first acquired my automatic skills, but Being Responsible required the ability to think strategically.
After reading Benito's rephrasing of the point...
Your identity/role, in near mode, seems to tell your System 1 what to be thinking, and your System 2 can change what identity/role you're running at a given time.
I immediately had the same reaction as Lumifer:
There is a trade-off, of course: your reaction is fixed by training and it's not necessarily appropriate to the specific situation.
A carefully crafted role (or set of roles) that allows your System 1 to function the way you want it to in most circumstances, while being a huge improvement for most, will still have problems if it is restricted only to System 1 responses. The appropriate reaction is sometimes to switch to System 2 where one can go through all the methods of rationality, not to just perform certain habits or responses that rationality has shown to be better than the default.
At first, this seemed contradictory to me, as many of the high-value things on LW seem to be System 2 processes. But then I realized that many of the habits of rationality involve training your System 1 to fire up your System 2 in the right circumstances and it seems that this could be crafted into the Roles that we try to adopt. For example, not "flinching away" from an Ugh Field. Noticing when you are about to do it and kicking over to System 2 to do something about it. If you see yourself as the kind of person who does this (and practice of course) then I don't see why that can't be a part of the Roles that you train.
Excellent article Eneasz!
As a student of Martial Arts, I especially enjoyed the crystal clarity of the first half of your submission. I would like to add, however, that it is the beginner’s lesson that teaches us to appreciate that it is possible to respond far more quickly and fluidly with a well rehearsed set of moves void of thought interference. The greater aspiration though and the life lesson for me is that in other areas of our lives, we should believe that everything will come together naturally based on the wide range of investments already made to date; but in order to reap the full benefits, we should attempt to be "without mind" to allow the spontaneous development of sequences across disciplines and even across roles.
I agree, there is that aspect of martial training that allows you to "recognise" which set of responses you should make in a given situation. Could you perhaps extend the analogy and consider how one would train to "recognise" which role one should be in?
There is a particular danger in "grokking" quick-response and that is - it's extremely hard to self-evaluate that you're doing it wrong, and it takes a lot of time to unlearn a particular. I attest to this as a professional musician and advanced modal interfaces afficionado.
Also, I'm doubting the optimality of "eliminating your slow, conscious deliberation" in any non-synthetic scenario (like most Martial Arts contests and most Martial Arts in general are). There's a reason Law Enoforcers do not act as Martial Artists, and draw a fine line between deliberating consciously and acting on their motoric training, and run a scenario through a specific rule set.
I'd rather see this article using Law Enforcement instead of Martial Arts as an analogy. It would plant it in the grounds of reality more thoroughly.
Thinking about what to do is an action in itself. If you pause to think whether to brake or steer left to avoid a crash, you're not doing either. If a SWAT officer pauses to think during the part of a raid when the most important decisions happen, people get shot.
Most optimal algorithms do not involve questioning their own validity. There are times when you design and optimize, and there are times when you execute. Downtime is only useful when you're not up.
Ran a phone call just now with my uncle who is a SWAT officer (well, localized equivalent of a SWAT officer). He says they're trained to run in two modes - decision-making and execution - and to switch the two routinely during any particular live-action scenario. He added, quote (paraphrased slightly because of translation), "those that run all the time in execute mode during live-action are morons that get people killed".
In car-crash scenario, reacting Fast only buys you time to react Properly, but indeed, it's what is needed. So, i'm here really moving the problem scope to other questions like:
There's no realistic situation where one should completely overtake the other. No, not even my own heartbeat is allowed to run in Execute mode all the time.
As a side note, the newest Robocop franchise installment kind of ventures to some depth (questionably) in this exact topic, and concludes that it's best to run on two modes at the same time, Decision-Maker being a slow tweaker of the Executor, detached for the most part. An overseer with only slight control.
I was only talking about the timeframe right after kicking down the door, when you really can't afford any delays in decision making, but there is only a very limited set of options you need to choose from. It's the training that gives you the options and the means to choose, you don't think them up on the spot.
In particular, they don't just stand there and think about the ethics of yelling at possibly innocent people while they may still be armed.
Similarly, in the car crash, there is no separate Proper and Fast reaction, because if the Fast one is not Proper, you're dead, end of story. You either make the split-second decision that saves your life or you don't, and whatever you think about afterwards is the result of already having made the correct choice.
By the time you get into a situation, you should already have a decent working model of your car's controls and capabilities in your current speed, the road, your surroundings etc. (a.k.a a driver's licence) so the fast path can reduce the problem into a small set of valid options and their exact execution instantly. What I'm willing to accept as Proper in this context is the act of learning to drive to the point where you don't have to signal the other drivers that you're still learning.
It seems pointless to me to ponder on proper ratios, your brain will abandon the illusion of conscious control when it deems necessary. There is no amount of conscious thought that will allow you to ponder on the many-worlds interpretation in freefall. I'm not quite sure what you mean by your heartbeat running in Execute mode, can you control it at will?
I think an excellent follow up article on this article would be "How to choose your roles wisely" or "How to develop roles in your psyche". In fact, the more I think about it, this has the potential to evolve into its own sequence if done carefully.
On the flip side, and this really is the million dollar question (and one that has been asked many many times on lesswrong), how to create the role of perfect agency? Now, it may be early to expect an answer to that, but certainly, if someone is up for writing that sequence it will be a step in the right direction.
how to create the role of perfect agency?
It's someone who takes "heroic responsibility". Source:
"You could call it heroic responsibility, maybe," Harry Potter said. "Not like the usual sort. It means that whatever happens, no matter what, it's always your fault. Even if you tell Professor McGonagall, she's not responsible for what happens, you are. Following the school rules isn't an excuse, someone else being in charge isn't an excuse, even trying your best isn't an excuse. There just aren't any excuses, you've got to get the job done no matter what. That's why I say you're not thinking responsibly, Hermione. Thinking that your job is done when you tell Professor McGonagall - that isn't heroine thinking. Like (SPOILER) is okay then, because it isn't your fault anymore. Being a heroine means your job isn't finished until you've done whatever it takes to (SPOILER), permanently. You can't think as if just following the rules means you've done your duty."
So, I guess watching superhero movies, and imagining yourself being one, could be a good start. Preferably movies where the superhero is not defined primarily by their supernatural powers, but by taking action where others don't. In other words, less Superman, more Batman.
Doing everything it takes to achieve some result, rather than just following the rules, creates perverse incentives for other people to slack off because they know that you will do whatever it takes.
It may still be a good idea when the consequences of not getting the result are so bad that even the negative effect of the perverse incentive isn't as bad, but that usually happens only with superheroes and Harry Potter-like characters.
Also, Batman is not so much defined by taking action, but by plot armor.
It can also create unsustainable expectations, which can be just as bad.
I once worked for a company that got bitten badly by this. Just after it left the startup stage, it got slapped with a contract that it realistically couldn't have met on time. Unwilling to accept this, the engineering team chose to do whatever it took to get the product out the door; one senior engineer in particular, a friend of mine, put in hundred-hour weeks to implement key features. And they succeeded. I probably wouldn't have gotten hired if they hadn't -- I came in just after this went down -- but that sparked a pattern of behavior that turned out to be the company's downfall.
Management interpreted this success not as the act of heroic effort that it was, but as a demonstration of engineering's regular capabilities. So the next few projects were scheduled just as tightly. We had some early successes, but burnout nipped at our heels throughout, and it finally caught up to my friend when work-related stress and lack of free time lost him his wife and his religion. (I'd been putting in crazy hours myself, but I was just out of college and didn't have as much to lose.) That proved lethal to the project; two deadlines slipped before it (nominally) got out of beta. The next project turned out similarly. After that we were given saner schedules, but the damage had been done; engineering didn't trust management, management didn't trust engineering, and the whole product line had taken a serious credibility hit.
Nice! I think you should include a disclaimer in your comment though. You only intuitively get it once try it for about 5 minutes. It seems pretty easy to diss, if you only give it a superficial reading...
This is why musicians play scales and arpeggios. Nobody really cares about scales or arpeggios. But when you run into a long sequence of them in some piece you are studying, it makes the difference between sight reading it and spending a week learning it.
FWIW, I think this is also why Design Patterns in SW Engineering is important. They are roles for programs.
I liked this post overall. Minor nitpick: I found the use of "guy who VERBs" to be a little jarring. Saying "person who VERBs" would be more inclusive.
That's a good point, and you're right. I wish "person" didn't feel so formal though. I'm having trouble thinking of a gender-neutral word that conveys the same casualness of "guy."
That sounds even more formal than "person" to me, actually.
Edit: how about "someone who acts"?
Perhaps I phrased my template too formally. Though as I search for examples, I notice that different uses of the word "guy" would require various replacements ("person," "someone," or "the one") in order to sound natural.
Really, I begin to think it would be simpler to alter our culture so that nobody expects "guy" to imply "male".
I've found training myself in a consequentialist/rationalist role to be pretty useful in my own life.
Not lying to myself in order to get myself to do things. Not pretending I can affect the past. Reminding myself that I want to believe what's true and that I want to do what's highest expected value. Doing the best with what I have, keeping my actual long-term goals in mind, listening to heavy metal music, and generally feeling like a badass.
Upvote, valuable insight. And meta thinking is switching from Tiger style to Crane style as the situation warrants. Good idea to have a set of modules ready to go.
Yeah, but the weak point is how to realize that a model switching is needed in that very moment.
Very interesting. I wonder how general the roles are. What you talk about at the end is basically bystander effect: I believe that different people are more or less vulnerable to that, and I wonder whether being more 'bystander' prone goes with being more likely to go along with pressure to conform (Millgram etc.) and possibly (to make it clear this isn't a straightforwardly ethical thing) more likely to collaborate in Prisoner's Dilemma. The most important role question might be simply whether you see yourself as a generalised Agent with responsibility for what actually happens beyond fulfilling set roles you've been given.
To quote HPMOR again: 'PC or NPC, that is the question'
I'd recommend a persona over a role. Make it a person to inhabit, instead of an idea to correspond to.
"The Guy Responsible for Shit" as a role is conceptual and abstract. If you're trying to program yourself, a complete person with full sensory detail would be more motivating. Picture an actual person, maybe yourself, as Responsible Guy, a superhero with a full back story and life and personality.
A role is a step more concrete than a verbal commandment, and better thereby. But an actual character is more concrete still.
I really enjoyed this post. Useful topic, some new insight that suggests other insights (as Gunnar_Zarncke pointed out), good examples.
This depends a lot on how you define "I". If you define it to be about deliberation then that means there no agency. There however no need to define deliberation as a central part of conscious action.
People meditate partly to be in a state where they aren't driven by mental deliberation. You frequently find that state described as being of a higher form of consciousness.
Role: vexatious litigator
Role: Amoralist
If you're committing the just world fallacy. Learn to check consequentialism.
Role: Conservative
Be cautious of great ideas. Even metamed went out of business
I would caution that the term "roles" should perhaps be modified if the intended connotation is positive to "adaptive roles" or "considered roles" to help avoid confusion with the idea that seems prevalent in the LW community, for good reason, that roles are generally not at all well calibrated to maximize utility
Recently I started to notice this kind of third-person perception of situations where I explain things that I understand well enough. The perception is more vivid when the topic is unusual, things I might've thought about a lot but only rarely discussed in similar contexts. I'm guessing explanation of a familiar topic can run mostly on System 1, freeing System 2 to observe the performance, and unfamiliar circumstances make the performance more interesting, hence perception of it more vivid.
A long time ago I thought that Martial Arts simply taught you how to fight – the right way to throw a punch, the best technique for blocking and countering an attack, etc. I thought training consisted of recognizing these attacks and choosing the correct responses more quickly, as well as simply faster/stronger physical execution of same. It was later that I learned that the entire purpose of martial arts is to train your body to react with minimal conscious deliberation, to remove “you” from the equation as much as possible.
The reason is of course that conscious thought is too slow. If you have to think about what you’re doing, you’ve already lost. It’s been said that if you had to think about walking to do it, you’d never make it across the room. Fighting is no different. (It isn’t just fighting either – anything that requires quick reaction suffers when exposed to conscious thought. I used to love Rock Band. One day when playing a particularly difficult guitar solo on expert I nailed 100%… except “I” didn’t do it at all. My eyes saw the notes, my hands executed them, and no where was I involved in the process. It was both exhilarating and creepy, and I basically dropped the game soon after.)
You’ve seen how long it takes a human to learn to walk effortlessly. That's a situation with a single constant force, an unmoving surface, no agents working against you, and minimal emotional agitation. No wonder it takes hundreds of hours, repeating the same basic movements over and over again, to attain even a basic level of martial mastery. To make your body react correctly without any thinking involved. When Neo says “I Know Kung Fu” he isn’t surprised that he now has knowledge he didn’t have before. He’s amazed that his body now reacts in the optimal manner when attacked without his involvement.
All of this is simply focusing on pure reaction time – it doesn’t even take into account the emotional terror of another human seeking to do violence to you. It doesn’t capture the indecision of how to respond, the paralysis of having to choose between outcomes which are all awful and you don’t know which will be worse, and the surge of hormones. The training of your body to respond without your involvement bypasses all of those obstacles as well.
This is the true strength of Martial Arts – eliminating your slow, conscious deliberation and acting while there is still time to do so.
Roles are the Martial Arts of Agency.
When one is well-trained in a certain Role, one defaults to certain prescribed actions immediately and confidently. I’ve acted as a guy standing around watching people faint in an overcrowded room, and I’ve acted as the guy telling people to clear the area. The difference was in one I had the role of Corporate Pleb, and the other I had the role of Guy Responsible For This Shit. You know the difference between the guy at the bar who breaks up a fight, and the guy who stands back and watches it happen? The former thinks of himself as the guy who stops fights. They could even be the same guy, on different nights. The role itself creates the actions, and it creates them as an immediate reflex. By the time corporate-me is done thinking “Huh, what’s this? Oh, this looks bad. Someone fainted? Wow, never seen that before. Damn, hope they’re OK. I should call 911.” enforcer-me has already yelled for the room to clear and whipped out a phone.
Roles are the difference between Hufflepuffs gawking when Neville tumbles off his broom (Protected), and Harry screaming “Wingardium Leviosa” (Protector). Draco insulted them afterwards, but it wasn’t a fair insult – they never had the slightest chance to react in time, given the role they were in. Roles are the difference between Minerva ordering Hagrid to stay with the children while she forms troll-hunting parties (Protector), and Harry standing around doing nothing while time slowly ticks away (Protected). Eventually he switched roles. But it took Agency to do so. It took time.
Agency is awesome. Half this site is devoted to becoming better at Agency. But Agency is slow. Roles allow real-time action under stress.
Agency has a place of course. Agency is what causes us to decide that Martial Arts training is important, that has us choose a Martial Art, and then continue to train month after month. Agency is what lets us decide which Roles we want to play, and practice the psychology and execution of those roles. But when the time for action is at hand, Agency is too slow. Ensure that you have trained enough for the next challenge, because it is the training that will see you through it, not your agenty conscious thinking.
As an aside, most major failures I’ve seen recently are when everyone assumed that someone else had the role of Guy In Charge If Shit Goes Down. I suggest that, in any gathering of rationalists, they begin the meeting by choosing one person to be Dictator In Extremis should something break. Doesn’t have to be the same person as whoever is leading. Would be best if it was someone comfortable in the role and/or with experience in it. But really there just needs to be one. Anyone.
cross-posted from my blog