Gunnar_Zarncke's Shortform

28 comments, sorted by Highlighting new comments since Today at 4:22 AM
New Comment

An Alignment Paradox: Experience from firms shows that higher levels of delegation work better (high level meaning fewer constraints for the agent). This is also very common practical advice for managers. I have also received this advice myself and seen this work in practice. There is even a management card game for it: Delegation Poker. This seems to be especially true in more unpredictable environments. Given that we have intelligent agents giving them higher degrees of freedom seems to imply more ways to cheat, defect, or ‘escape’. Even more so in environments that can be controlled to lesser degrees. How can that be true? What is making this work and can some underlying principle be found that would allow this to be applied to AI?

Most people are naturally pro-social. (No, this can't be applied to AI.) Given a task, they will try to do it well, especially if they feel like their results are noticed and appreciated.

A cynical hypothesis is that most of the things managers do are actively harmful to the project; they are interfering with the employees trying to do their work. The less the manager does, the better the chances of the project. "Delegation" is simply when manager stops actively hurting the project and allows others to do their best.

The reason for this is that most of the time, there is no actually useful work for the manager. The sane thing would be to simply sit down and relax, and wait for another opportunity for useful intervention to arise. Unfortunately, this is not an option, because doing this would most likely get the manager fired. Therefore managers create bullshit work for themselves. Unfortunately, by the nature of their work, this implies creating bullshit work for others. In addition to this, we have the corrupted human hardware, with some managers enjoying power trips and/or believing they know everything better than people below them in the hierarchy.

When you create a manager role in your company, it easily becomes a lost purpose after the original problems are solved but the manager wants to keep their job.

Most people are naturally pro-social. 


(No, this can't be applied to AI.) 


I don't like cynical views and while I have encountered politics and seen such cases I don't think that paints a realistic view. But I will run with your cynical view and you won't like it ;-)

So we have these egotistical managers that only want to keep their job and raise in ranks. Much closer to non-social AI, right? How come more delegation works better for them too?   

Mind you, I might be wrong and it works less and less the further up you go. It might be that you are right and this works only because people have enough social behavior hard-wired that makes delegation work. 

But I have another theory: Limited processing capacity + Peter Principle.

It makes sense to delegate more - especially in unpredictable environments - because that reduces your processing load of dealing with all the challenging tasks and moves it to your subordinates. This leaves less capacity for them to schema against you and gives you the capacity to scheme against your superior. Und so up the chain. Capable subordinates that can deal with all the stuff you throw at them have to be promoted so they have more work to do until they reach capacity too. So sometimes the smart move is to refuse promotion :-)

I guess we agree that limited processing capacity means that interfering with the work of your underlings -- assuming they are competent and spending enough of their processing capacity on their tasks -- is probably a bad move. It means taking the decision away from the person who spends 8 hours a day thinking about the problem, and assigning it to a person who spent 30 seconds matching the situation to the nearest cliche, because that's all they had time for between the meetings.

This might work if the person is such a great expert that their 30 seconds are still extremely valuable. That certainly is possible; someone with lots of experience might immediately recognize a frequently-made mistake. It is also is the kind of assumption that Dunning and Kruger would enjoy researching.

I might be wrong and it works less and less the further up you go

That would make sense. When you are a lowest-level manager, if you stop interfering, it allows the people at the bottom to focus on their object-level tasks. But if you are a higher-level manager, how you interact with the managers below you does not have a direct impact on the people at the bottom. Maybe you manage your underlings less, and they copy your example and give more freedom to the people at the bottom... or maybe you just gave them more time to interfere.

So sometimes the smart move is to refuse promotion

So you have more time to scheme... but you have to stay low in the pyramid. Not sure what you scheme about then. (Trying to get to the top in one huge jump? Sounds unlikely.)

Have you ever managed or worked closely with great team-leads? 

I was a team leader twice. The first time it happened by accident. There was a team leader, three developers (me one of them), and a small project was specified. On the first day, something very urgent happened (I don't remember what), the supposed leader was re-assigned to something else, and we three were left without supervision for unspecified time period. Being the oldest and most experienced person in the room, I took initiative and asked: "so, guys, as I see it, we use an existing database, so what needs to be done is: back-end code, front-end code, and some stylesheets; anyone has a preference which part he would like to do?" And luckily, each of us wanted to do a different part. So the work was split, we agreed on mutual interfaces, and everyone did his part. It was nice and relaxed environment: everyone working alone at their own speed, debating work only as needed, and having some friendly work-unrelated chat during breaks.

In three months we had the project completed; everyone was surprised. The company management assumed that we will only "warm up" during those three months, and when the original leader returns, he will lead us to the glorious results. (In a parallel Everett branch, where he returned shortly before we finished the product, I wonder whether he got a bonus and promotion.) Then everything returned to normal: more micromanagement, lower productivity, people burned out.

The second time, we were a small group working together for some time already. Then our manager quit. No one knew who would get the role next, and in an attempt to deflect a possible danger, I volunteered to do it on top of my usual work. What happened was that everyone worked exactly the same as they did before, only without the interruptions and extra stress caused by management, and I got some extra paperwork which I gradually reduced to minimum. The work progressed so well -- no problems, no complaints from users, the few tickets we got almost always turned out to be a problem outside our project -- that higher management concluded that there is apparently too litle work to do on our project, so the team members were assigned to also work on extra projects in parallel.

Perhaps my short experience is not representative, but it suggests that a manager, merely by not existing, could already create a top-decile work environment in terms of both work satisfaction and productivity. The recommended mantra to recite every day is: "first, do no harm". My experience also suggests that this approach will ultimately get punished, despite the increased productivity: the expected outcome is more work for no pay raise until you break, or just being told to return to the old ways without any explanation why. I assume I am missing some crucial maze-navigating skills; for someone trying to be a professional manager this would be fatal; luckily I do not have this ambition.

It is quite possible that this approach only works when there is a good team: in both cases I worked with people who were nice above average. If you had a dominant asshole in the team, this could easily become a disaster: the power vacuum left by a passive manager would simply be replaced by an ambitious alpha male, who would probably soon be promoted into the role of formal leader. So perhaps the companies play it safe by using a widely applicable strategy that happens to be inferior in the case of good employees who also happen to be good people; quite likely this is because the companies simply cannot recognize such people.

Is there a leadership level beyond this? Sure, but in my quarter century of career I have only met such manager once. What he did was basically meeting each of his people once a day in the morning (this was long before I heard about "daily standups" and such) and talking with him for 5 or 10 minutes; with each team member separately, in the manager's room. He asked the usual questions "what did you do yesterday?", "what is your plan for today?", "are there any obstacles to your work?", but there was zero judgment, even if you said things like "yesterday I had a really bad day, I tried some things but at the end it was wrong and I had to throw it all away, so today I am starting from scratch again"; essentially he treated you like an adult person and assumed that whatever you did, there was a good reason for that. Before and after the report, a very short small talk; it helped that he was extremely intelligent and charismatic, so for many people this was the best part of the day. Also, the obstacles in work that you mentioned, he actually did something about them during the day, and always reported the outcome to you the next morning. Shortly, for the first and the last time so far in my career, I had a regular feeling that someone listens to me and cares about what I do (as opposed to just whipping me to run faster in order to meet arbitrary deadlines, randomly interrupting me for no good reason, second-guessing my expert opinion, etc.).

So yes, there is a level beyond "not doing harm" and it is called "actually motivating and helping", but I guess most managers dramatically overestimate their ability to do it... and when they try regardless, and ignore the feedback, they actively do harm.

Thank you a lot. Your detailed account really helps me understand your perspective much better now. I can relate to your experience in teams where micromanagement slows things down and prevents actually relevant solutions. I have been in such teams. I can also relate to it being advantageous when a leader of questionable value is absent. I have been in such a team too - though it didn't have such big advantages as in your case. That was mostly because this team was part of a bigger organization and platform where multiple teams had to work together to something done, e.g. agree on interfaces with other teams. And in the absence of clear joint goals that didn't happen. Now you could argue that then the management one level up was not doing its job well and I agree. But the absence of that management wouldn't have helped either - it could have led to a) each team trying to solve some part of the problem. It could have led to b) some people from both teams getting together and agreeing on interfaces and joining goals or it could have led to c) the teams agreeing on some coordination for both teams. a) in most cases leads to some degree of chaos and failure and b) establishes some kind of leadership on the team level (like you did in your first example) and c) results over time in some leadership one level up. I'd argue that some kind of coordination structure is needed. Where did the project you did implement in your first case come from? Somebody figure out that it would provide value to the company. Otherwise, you might have built a beautiful project that didn't actually provide value. I think we agree that the company you worked in did have some management that provided value (I hope it was no moral maze). And I agree that a lot of managers do not add value and sometimes decrease it. On the other hand, I have worked for great team leads and professional managers. People who would listen, let us make our own decisions, give clear goals but also limits, help, and reduce impediments. This is really not a secret art. The principles are well-known (for a funny summary see e.g. Leadersheep). But it turns out that building a big organization is hard. Politics is real and professional management is still mostly a craft. It rarely approaches something you can call engineering much less hard science. And I am looking for that. That's part of why I wrote this shortform on processes and roles. Everybody is just cooking with water and actual organization structures often leave something to be desired. I guess that's why we do see extraordinary companies like Amazon sometimes - that hit on a sweet spot. But by talent or luck, not by science. And the others have to make do with inadequate solutions. Including the managers of which you maybe saw more than I did. 


this team was part of a bigger organization and platform where multiple teams had to work together to something done, e.g. agree on interfaces with other teams. And in the absence of clear joint goals that didn't happen.

I have seen this happen also in a small team. Two or three guys started building each his own part independently, then it turned out those parts could not be put together; each of them insisted that others change their code to fit his API, and refused to make the smallest change in his API. It became a status fight that took a few days. (I don't remember how it was resolved.)

In another company, there was a department that took care of everyone's servers. Our test server crashed almost every day and had to be restarted manually; we had to file a ticket and wait (if it was after 4PM, the server was restarted only the next morning) because we did not have the permission to reset the server ourselves. It was driving us crazy; we had a dedicated team of testers, and half of the time they were just waiting for the server to be restarted; then the week before delivery we all worked overtime... that is, until the moment the server crashed again, then we filed the ticket and went home. We begged our manager to let us pool two hundred bucks and buy a notebook that we could turn into an alternative testing environment under our control, but of course that would be completely against company policy. Their manager refused to do anything about it; from their perspective, it meant they had every day one support ticket successfully closed by merely clicking a button; wonderful metric! From the perspective of our manager's manager, it was a word against a word, one word coming from the team with great metrics and therefore more trustworthy. (The situation never got solved, as far as I know.)

...I should probably write a book one day. Except that no one would ever hire me afterwards. So maybe after I get retired...

So, yes, there are situations that require to be solved by greater power. In long term it might even make sense to fire a few people, but the problem is that these often seem to be the most productive ones, because other people are slowed down by the problems they cause.

Where did the project you did implement in your first case come from? Somebody figure out that it would provide value to the company. Otherwise, you might have built a beautiful project that didn't actually provide value. I think we agree that the company you worked in did have some management that provided value (I hope it was no moral maze).

Yeah, but we have two different meanings of the word "management" here. Someone who decides which project to do -- this is useful and necessary. Or someone who interrupts you every day while you are trying to work on that project -- I can imagine that in some teams this may also be necessary, but arguably then your problem is the team you have (at least some parts of it). Motte and bailey of management, sort of.

From epistemic perspective, I guess the problem is that if you keep micro-managing people all the time, you can never learn whether your activity actually adds or removes value, simply because there is nothing to compare to. (I guess the usual null hypothesis is "nobody ever does anything", which of course make any management seem useful; but is it true?) Looking at the incentives and power relations, the employee at the bottom doesn't have an opportunity to prove they could work just as well without the micro-management, and the manager doesn't have an incentive to allow the experiment. There is also the "heads I win, tail you lose" aspect where bad employee performance is interpreted as necessity of more management, but good employee performance is interpreted as good management, so either way management is perceived as needed.

This is really not a secret art. The principles are well-known (for a funny summary see e.g. Leadersheep).

Yep. That's a very good summary. Heh, I fail hard at step 1 (creating, or rather communicating a strong vision).

But it turns out that building a big organization is hard. Politics is real and professional management is still mostly a craft. It rarely approaches something you can call engineering much less hard science.

Seems analogical to social sciences: in theory, they are much more difficult than math or physics, so it would make sense if smarter people studied them; in practice, it's the other way round, because if something is too difficult to do properly, it becomes easy to bullshit your way to the top, and intelligent people switch to something where being intelligent gives you a clear comparative advantage.

Good luck to you! I suppose your chances will depend on how much autonomy you get; it is hard to do things right, if the sources of problem are beyond your control. However, if you become a great manager and your people will like you, perhaps in the future you can start your own company and give them a call whether they would like to work for you again.

Thank you. I agree with your view. Motte and bailey of management yep. I especially liked this:

Seems analogical to social sciences: in theory, they are much more difficult than math or physics, so it would make sense if smarter people studied them; in practice, it's the other way round, because if something is too difficult to do properly, it becomes easy to bullshit your way to the top, and intelligent people switch to something where being intelligent gives you a clear comparative advantage.

It turns out that the alignment problem has some known solutions in the human case. First, there is an interesting special case namely where there are no decisions (or only a limited number of fully accounted for decisions) for the intelligent agent to be made - basically throwing all decision-making capabilities out of the window and only using object recognition and motion control (to use technical terms). With such an agent (we might call it zero-decision agent or zero-agent) scientific methods could be applied on all details of the work process and high efficiency reached: Scientific Management (also known as Taylorism). Obviously the unions hated it and it was later outlawed. I think something might be learned from this approach for AI control: Maybe we can build on top of a known solution for a zero-agent and prove that certain known decision classes are also safe. 

Maybe other insights from management theory - which after all is about aligning intelligent agents - could also transfer. The alignment problem is called Principal-Agent Problem in the literature and there are quite a few Solutions to Principal-Agent Problems in Firms (Gary Miller 2005). The approaches should sound familiar: Solutions Based on 

  • Incentives Linked io Agent Outcomes,
  • Direct Monitoring of Agent Actions,
  • Cooperation Between Principal and Agent, and
  • Cooperation within Teams

Tangentially related on LessWrong: The AI Alignment Problem has already been solved once 

Obviously the unions hated it and it was later outlawed.

I wonder how could one outlaw a thing like this. Suppose that most managers believe that Taylorism works, but it is illegal to use it (under that name). Wouldn't they simply reintroduce the practices, step by step, under a different name? I mean, if you use a different name, different keywords, different rationalization, and introduce it in small steps, it's no longer the same thing, right? It just becomes "industry standards". (If there happens to be an exact definition, of course, this only becomes an exercise how close to the forbidden thing you can legally get.)

From the Wikipedia article, I got the impression that what was made illegal was the use of stop-watch. Okay, so instead of measuring how many seconds you need to make a widget, I am going to measure how many widgets you make each day -- that is legal, right? The main difference is that you can take a break, assuming it will allow you to work faster afterwards. Which may be quite an important difference. It this what it is about?

Wouldn't they simply reintroduce the practices, step by step, under a different name?

I assume that that's what happened. Some ideas from scientific management were taken and applied in less extreme ways.

I think there's something here, but it's usually thought of the other way around, i.e. solving AI alignment implies solving human alignment, but the opposite is not necessarily true because humans are less general intelligences than AI.

Also, consider that your example of Taylorism is a case study in an alignment mechanism failing, in that it tried to align the org but failed in that it spawned the creation of a subagent (the union) that caused it to do something management might have considered worse than the loss of potential gains given up by not applying Taylorism.

Anyway, this is a topic that's come up a few times on LessWrong; I don't have links handy though but you should be able to find them via search.

I'm not trying to prove full alignment from these. It is more like a) a case study at actual efforts to align intelligent agents by formal means and b) the identification of conditions where this does succeed.

Regarding its failure: It seems that a close reading of its history doesn't prove that: a) Taylorism didn't fail within the factories and b) the unions were not founded within these factories (by their workers) but existed before and pursued their own agendas. Clearly real humans have a life outside of factories and can use that to coordinate - something that wouldn't hold for a zero-agent AI. 

I tried to find examples on LW and elsewhere. That is what turned up the link at the bottom. I am on LW for quite a while and have not seen this discussed in this way. I have searched again and all searches involving combinations of human intelligence, alignment and misc words for analogy or comparison turn up not much than this one which matches just because of its size:

Can you suggest better ones?

Thank you for your detailed reply. I was already wondering whether anybody saw these shortform posts at all. They were promoted at a time but currently it seems hard to notice them with the current UI. How did you spot this post?

I read LW via /allPosts and they show up there for me. Not sure if that's the default or not since you can configure the feed, which I'm sure I've done some of but I can't remember what.

The /allPosts is pretty useful. Thank you!

Team Flow Is a Unique Brain State Associated with Enhanced Information Integration and Interbrain Synchrony

It's also possible to experience 'team flow,' such as when playing music together, competing in a sports team, or perhaps gaming. In such a state, we seem to have an intuitive understanding with others as we jointly complete the task at hand. An international team of neuroscientists now thinks they have uncovered the neural states unique to team flow, and it appears that these differ both from the flow states we experience as individuals, and from the neural states typically associated with social interaction.

Researchers found increased beta and gamma brain wave activity in the left middle temporal cortex. This region of the brain is typically associated with information integration and key functions like attention, memory, and awareness, which are "consistent with higher team interactions and enhancing many flow dimensions," the team writes. However, what was unique about team flow, was that participants' neural activity appeared to synchronize. When participants were performing the task as a unit, their brains would mutually align in their neural oscillations (beta and gamma activity), creating a "hyper-cognitive state between the team members." If brains can be functionally connected through inter-brain synchrony, does this mean it is not only our brain that contributes to our consciousness? It's a curious question, but the authors warn it is much too soon to tell. "Based on our findings, we cannot conclude that the high value of integrated information correlates with a modified form of consciousness, for instance, 'team consciousness'," they write. "Its consistency with neural synchrony raises intriguing and empirical questions related to inter-brain synchrony and information integration and altered state of consciousness."

Found via good old Slashdot.

Insights about branding, advertising, and marketing.

It is a link that was posted internally by our brand expert and that I found full of insights into human nature and persuasion. It is a summary of the book How Not to Plan: 66 Ways to Screw it Up:

(I'm unaffiliated)

Roles serve many functions in society. In this sequence, I will focus primarily on labor-sharing roles, i.e. roles that serve splitting up productive functions as opposed to imaginary roles e.g. in theater or play. Examples of these roles are (ordered roughly by how specific they are):

  • Parent
  • Engineer (any kind of general type of job)
  • Battery Electronics Engineer (any kind of specific job description)
  • Chairman of a society/club
  • Manager for a certain (type of) project in a company
  • Member in an online community
  • Scrum master in an agile team
  • Note-taker in a meeting

You probably know the note-taker role. The tasks to be performed are: Writing down key discussion points and decisions, asking questions in cases where the points discussed were not clear, and sending around the notes to all participants of the meeting. The goal is usually to keep a record to consult later. Additionally, the note-taker is usually expected to be neural. Other expectations might be how timely the notes are sent around.

Some examples of what is commonly called a role but not in our strict labor-sharing sense:

  • Children playing father, mother, kid or playing animals
  • Role acting e.g. in theater (where the term role originates)

And some non-examples for roles in the labor-sharing world: 

  • Participant in a conversation
  • Responsible for organizing a specific meeting
  • Responsible for a specific post in a forum

Borderline case: Founder of a company. There is little pre-existing knowledge of what founders can or should do in general. In their company expectations on them will evolve quickly and taking on and off the role is not easy.

Roles are important. This shortform is telling you why. An example: The role of a moderator in an online forum. The person (in the following called agent) acting in this role is expected to perform certain tasks - promote content, ban trolls - for the benefit of the forum. Additionally, the agent is also expected to observe limits on these tasks e.g. to refrain from promoting friends or their own content. The owners of the forum and also the community overall effectively delegate powers to the agent and expect alignment with the goals of the forum. This is an alignment problem that has existed forever. How is it usually solved? How do groups of people or single principals use roles to successfully delegate power? 

Philosophy with Children - In Other People's Shoes

"Assume you promised your aunt to play with your nieces while she goes shopping and your friend calls and invites you to something you'd really like to do. What do you do?"

This was the first question I asked my two oldest sons this evening as part of the bedtime ritual. I had read about Constructive Development Theory and wondered if and how well they could place themselves in other persons' shoes and what played a role in their decision. How they'd deal with it. A good occasion to have some philosophical talk. This is the (shortened) dialog that ensued:

The immediate answer by A: "I will watch after the girls."

Me: "Why?"

A: "Because I promised it."

B: "Does A also promise it and get a call?"

Me: "This is about your nieces and your friend, not about your brother."

B: "But I need this for my answer."

Me: "I don't see why, but OK, assume that he is not involved."

B: "Because I would ask him whether he might play with the girls in exchange for a favor."

Me: "OK, but please assume that he is away."

B: "Then I could ask my aunt whether somebody else can watch for the girls or whether I could do it together with my friend."

Me: "Please assume that she doesn't find somebody and that she doesn't want somebody she doesn't know in her house."

B: "Then I'd do it."

Me: "Why?"

B: "Because I promised it. I'd tell my friend that we can do it another time."

We had another scenario: "Imagine that you and a fellow pupil C are guests at a friend and having a meal. You know that C is from a family that is very strict about not eating a kind of food that you like very much. Would you advise C to eat it or not?"

A (quickly): "I'd advise to not eat it."

Me: "Why?"

A: "I like rules."

B (after some consideration): "I'd advise to follow their heart."

Me: "And if you were C?"

B: "I'd at least try a bit."

(this was followed with a discussion about possible long-term consequences)

For me, it was still not clear whether this implied whether he followed only his preferences considered this in the context of the rules in the family. So I proposed a setting where he had to imagine being in another country with different laws. We settled on a rule he accepts here (indemnification) but that was much harsher in the other country. He asked whether he had the same feelings as here which after some clarification I confirmed. He argued that he wouldn't like the rule in the other country because it set questionable incentives: "If the punishment is that strong that tells people that it is OK to punish equally strong normally."

Philosophy with Children - Mental Images 

One time my oldest son asked me to test his imagination. Apparently, he had played around with it and wanted some outside input to learn more about what he could do. We had talked about before and I knew that he could picture moving scenes composed of known images. So I suggested

  • a five with green white stripes - diagonally. That took some time - apparently, the green was difficult for some reason, he had to converge there from black via dark-green
  • three mice
  • three mice, one yellow, one red, and one green
  • the three colored mice running behind each other in circles (all no problem)
  • he himself
  • he himself in a mirror looking from behind (no problem)
  • two almost parallel mirrors with him in between (he claimed to see his image infinitely repeated; I think he just recalled such an experiment we did another time).
  • a street corner with him on the one side and a bike leaning on the other wall with the handlebar facing the corner and with a bicycle bell on the left side such that he cannot see the bike.
  • dito with him looking into a mirror held before him so he can see the bike behind the corner.

The latter took quite some time, partly because he had to assign colors and such so that he could fully picture this and then the image in the mirror. I checked by asking where the handlebar is and the bell. I had significant difficulties imagining this and correctly place the bell. I noticed that it is easier to just see the bell once the image in the mirror has gained enough detail (the walls before and behind me, the corner, the bike leaning on the corner, the handlebar).

I also asked for a square circle which got the immediate reply that it is logically impossible.

If you have difficulties doing these (are judge them trivial): This is one area where human experience varies a lot. So this is not intended to provide a reference point in ability but an approach to teach human difference, reflection and yes also practice imagination - a useful tool if you have it. If not you might be interested in what universal human experiences are you missing without realizing it.

Origins of Roles 

The origin of the word role is in the early 17th century: from French rôle, from obsolete French roule ‘roll’, referring originally to the roll of paper on which the actor's part was written (the same is the case in other languages e.g. German).

The concept of a role you can take on and off might not have existed in general use long before that. I am uncertain about this thesis but from the evidence I have seen so far, I think this role concept could be the result of the adaptations to the increasing division of labor. Before that people sure were behaving sometimes as a father, carpenter, or soldier - but I contend not because they were consciously switching roles (though some individuals may have reflected sufficiently I presume there was no common knowledge or language for ‘role switching’). The only persons regularly in a different role were actors. It may be that in earlier times it was even necessary to wear a mask to allow actors to take on a role convincingly. To help their audience who otherwise might not have made the mental leap of treating them differently from the acted character.

The Cognitive Range of Roles

A role works from a range of abstraction between professions and automation. In a profession one person masters all the mental and physical aspects of trade and can apply them holistically from small details of handling material imperfections to the organization of the guild. At the border to automation, a worker is reduced to an executor of not yet automated tasks. The expectations on a master craftsman are much more complex than on an assembly-line worker.

With more things getting automated this frees the capacity to automate more complex skills. And so on. This is seen in real-time with software: Today a lot of software is (still) built semi-manually (some build tool is started to compile and bundle files together to some executable or installer). Every developer knows how to do this. It is a simple role to take on: “Can you do the build, please?” As the company and the software grow there is a phase where either this becomes a more fleshed-out role, “build master”, maybe even a dedicated job position - or more likely it will be automated. And as the automation - the build process or “pipeline” - becomes more complicated and you need the role of a build software expert. At this point, some tasks previously requiring skill and experience have been standardized and automated and what remains is the more complex task of managing the automation.

Over time, the range between what is fully automatable and what humans can do in an unpredictable environment, shrinks. With AGI there will be no roles left for humans. At least no roles in the sense used here. I think low skilled people already feel this.

When trying to get an overview of what is considered a role I made this table:

Type of roleExamplePurposeDistinctionCDF Level
(Children's) play actingCop, FatherPlay, imitation, learningShallow copy, present in higher animalsImpulsive (1) to Instrumental (2)
Social roleMother, HusbandElementary social function Since the ancestral environment, closely moderated by biological functionInstrumental (2) to Socialized (3)
Occupation (rarely called role but shares traits)CarpenterGetting things done in a simple societyRarely changed, advancement possible (apprentice to master)Socialized (3)
A role in (stage) acting e.g. theaterHamletEntertainment, education, cultural transmissionPeople know that this is a fiction, to make this explicit masks may be usedSocialized (3) to Self-authoring (4)
A role in a processModeratorGetting things done in a complex societyRoles can be taken on and given up, while in a role an agent’s actions are measured relative to the role Socialized (3) to Self-authoring (4)


CTF refers to Constructive Developmental Framework as explained e.g. here (nice pics) and also on LW here.

I'm very unsure about the levels I assigned. Also, I got feedback that the degree to which roles can be changed and were changed may be different from how I interpret the historical record. 

In any sizable organization, you can find a lot of roles. And a lot of people filling these roles - often multiple ones on the same day. Why do we use so many and fine-grained roles? Why don’t we continue with the coarse-grained and more stable occupations? Because the world got more complicated and everybody got more specialized and roles help with that. Division of labor means breaking down work previously done by one person into smaller parts that are done repeatedly in the same way - and can be assigned to actors: “You are now the widget-maker.” This works best when the tasks are easy to learn so it is easy to find someone to do it. But as humans are not plug-compatible and actual requirements may vary so there is always training required that can be amortized over repeatedly performing the task - a role. So roles make sense structurally - but why do people actually do what is expected of them and don’t just follow their own agenda? This an alignment problem - in this case between the organization and the agent - and we might learn something about the AI alignment problem from it.

What are the common aspects of these labor-sharing roles (in the following called simply roles)? 

One common property of a role is that there is common knowledge by the involved persons about the role. Primarily, this shared understanding is about the tasks that can be expected to be performed by the agent acting in the role as well as about the goals to be achieved, and limits to be observed as well other expectations. These expectations are usually already common knowledge long beforehand or they are established when the agent takes on the role.

The second important aspect of a role is that it can be taken on and off or assigned. Sometimes this happens explicitly when an employee is designated or volunteers to be a moderator. Sometimes this happens automatically or habitually to perform certain actions e.g. when a forum moderator writes a comment as a regular poster or when a developer assumes root permissions (sudo lecture).

Other aspects are of interest in the social sciences e.g. interactions between roles or the social function of roles