I've often thought that seniority/credential based hierarchies are stable and prevalent both because they benefit those already in power, and they provide a defined, predictable path for low status members to become high status. One is more motivated to contribute and support a system that guarantees them high status after X years if they are of middling competence, rather than a system that requires them to be among the best at some quantifiable metric. The longer someone spends in a company, the more invested they become in their relative position in the company rather than the company's absolute success, and if the company has gotten "too big to fail", it's much more predictably personally beneficial to prioritize personal relative status since the company will do well either way.
In the case of a very competent generalist, who isn't demonstrably loyal, the very world itself (over which they are able to exert optimization power) will seem to ally itself with them. If you are a middlingly competent leader-by-luck, such an individual will of course be a threat to your position of power, unless your power/welfare is something the compent general optimizer is choosing to optimize for. So... it's the alignment problem all over again. Human-to-human alignment costs us a lot of social overhead.
You forgot the most fundamental option, decide based on who makes the most profit/acquires the most fungible resources. Which is what nearly all of the outlier private organizations do.
This can't applied everywhere though, especially where the lag between decision and result is multiple decades.
The most famous example in the financial world is RenTech. You could be the most anodyne guy in the world personally, with nothing super special going for you, but if you can land a position as an anonymous intern and consistently make above average profits week after week, they will promote very quickly into a seven figures total compensation position.
Maybe even before age 30 depending on just how above average the performance is.
It's probably the most exceptional example of organizing via pure merit. You can't quite just walk in and by year's end start earning the equivalent of a sizable house in Berkely every quarter, but it's not that far off.
At that level all these games mentioned in the post simply peter out once folks figure out it's literally 10x slower and 10x less reliable than just shutting up and earning profits.
It's true though for those who don't have that kind of competence, or who are in the situations with too much delay between decisions to assess, that all the various tricks and schemes start to come into play.
This is because moderately above average people are actually quite tough to rank and distinguish based on anything other then hard to change physical characteristics.
i.e. nearly everyone at the 95th percentile has a similar enough chance of being a competent employee once promoted to middle management that noise, random chance, etc., will drown out any actual differences.
Probably even RenTech has a non-zero amount of mis-promotions and mis-hires every year due to happenstance.
decide based on who makes the most profit/acquires the most fungible resources
...within some period of evaluation. That's how you get Jack Welch hollowing out GE. That's how you get managers who raise profits by cutting out maintenance and training, and hope to get promoted fast enough to outrun the resulting problems so they can blame their successor.
The most famous example in the financial world is RenTech
Their main fund results are not nearly as good, but they got investment largely using their famous internal-only Medallion fund as advertising. I've seen some people say that famous Medallion fund got much of its good returns by sacrificing their main fund performance, by doing trades then deciding which fund those belonged to post-hoc.
There's also a lot of insider trading going on at big hedge funds, which is totally legal as long as you pool transactions together such that you only ever don't do transactions based on insider info. What's illegal is doing transactions based on insider info. That's part of why you see siloed trading groups so often.
Sigh. I've seen a lot of RenTech worship and it rather annoys me.
Huh? I'm explicitly not 'worshipping' them. Did you skip reading the latter half of my comment?
I'm pretty sure most passing readers will already interpret 'mis-hire' and 'mis-promotion' as implying some degree of destructive efforts.
far more than a highly effective not non-technical MBA would be
Is this meant to have only a single negation? I'm not sure how the sentence as written works in context.
Yeah, sounds right, though I've encountered hierarchy resistant organizations that successfully route task by capabilities. It's currently quite fragile but I don't see any fundamental reason it has to stay that way. A non-concentrateable currency creation system such as grassroots economics and a cultural focus on credit assignment towards technical workers can move towards keeping the graph in a mesh structure, some would say the numbers are optional in small orgs, sometimes even big ones, compare types of open source culture. The difficulty is ensuring that any numerical tracking of predictions about who will do what well and by whose request, stays precise enough, without getting precise enough for the people who want to exploit politics-like vulnerabilities to be able to do so.
This feels like less of an engineer vs management conflict, and more of a technically-focused vs organization-focused management conflict. The nuance, of course, is that in smaller orgs and startups, technical expertise is by far the most important element, but in larger groups, both are needed, in different quantities at different levels of organizational abstraction.
From what I've seen, there are different balances even across similar-seeming orgs, but at lower levels, technical expertise dominates, at the VERY top, social and organizational expertise dominates, and there's a gradient in between. NOBODY gets very far without understanding elements of both, and without being able to communicate well in both domains. Those who rise (in org level) are generally engineers (or at least engineering-competent) who learn to excel in the other domains - the reverse is much less common.
I think a lot of this kind of analysis makes a big mistake to think of people and roles as static, with a simple(-ish) sortation to how much respect/compensation they get. In fact, individuals grow and change over time (with some intent to be more useful or a better fit to the job(s) they want), and organizations change their culture and needs to fit with their changing personnel. Both of these are relatively slow, but they really do matter.
in smaller orgs and startups, technical expertise is by far the most important element
Not necessarily. It depends on the organization. For example, consider a small lobbying company in Washington DC.
at lower levels, technical expertise dominates, at the VERY top, social and organizational expertise dominates, and there's a gradient in between
Part of the point of my post was that those aren't actual categories that exist. "Social expertise" is something that exists in a certain context and culture. There are different groups that communicate well internally, but each consider the others to have "poor social skills". You can't just treat cultures and expertise like they're stat points in an RPG.
I also think that in the minds of management, the more general a technical expert is, the more of a potential threat they are, and thus loyalty becomes even more important. If that's the case, it might be sometimes beneficial to pretend to be ignorant of areas outside your specialty, especially the business context of your technical work. (That's pretty inconvenient for someone like me.)
Curious what you meant by "someone like me". Are you referring to your job role, disposition, something else?
Sometimes, a group in some organization has a highly technical and highly effective leader. Kelly Johnson (Skunk Works) and Hyman Rickover (US Navy nuclear propulsion) are famous examples. A naive economist might expect such people to be well-liked by management above them, because their skills are good for the organization and complementary to those of non-technical managers. That's not what we generally see in reality.
In my experience, and in the stories I've heard, such technical leaders are especially disliked by upper management, far more than a highly effective non-technical MBA would be. I've even been told that unique competence being noticed by upper managment is a negative for career prospects in that situation.
Why would that be the case? The only explanation that makes sense to me is that effective technical managers are considered a threat by management above them - but why would they be more of a threat than a MBA who talks the business talk?
There are some cultural differences between engineers and non-technical managers, but I don't think that's an explanation. One reason is, technical leaders can find allies even higher up that support them. For example, Rickover had allies in Congress, and that's the only reason he wasn't pushed out...until he got pushed out by John Lehman, a Ph.D. in American foreign policy who's worked as an investment banker. Leslie Groves was almost pushed out in 1927, but Major General Edgar Jadwin interceded and noted that Groves's superiors were at fault for the problems blamed on him - that was a guy 5 ranks above Groves in the Army.
My current view is that politician-type managers and engineer-type managers naturally form opposing coalitions. They each favor people of the same type, and try to push local organization norms in different directions.
In America, today, politican-type managers have won conclusively almost everywhere. I've actually seen some of a conflict between such coalitions play out once, and I'd say it's an even match when the groups are equal in size and nobody else is involved. One group backstabs and fights over social dominance like high school girls, but that's balanced out by the other group spending half their time arguing about who's smarter and the other half arguing about emacs vs vim; the underlying dynamic is the same, but one group has the pretense of authority being tied to intelligence, and has a greater tendency to argue about actions instead of personnel selection.
Normies prefer business speak to technobabble, while nature has the opposite preference, and so the balance is tipped depending on which is more relevant.
I am, of course, exaggerating somewhat, and all groups have some overlap in their tendencies. There are also other ways to organize hierarchy, such as:
That last one is perhaps my least-favorite. Anyway, we see a similar effect, where organizations based around seniority are all-in on seniority, because the people who want that to be the principle are in charge. There's always a social hierarchy, so in a sense the only choice society and leaders get to make is what's used as its basis.
Companies often have distinctive corporate culture, but companies have to interact with each other, and people move between them, so there's some pressure towards homogenization. In America, ongoing consolidation has been towards what's described in Moral Mazes. From inside the system, it might seem like the only possible system, but take heart, for alternatives are possible - for example:
It's possible to make an engineering-focused startup. You just have smart engineers controlling the leadership positions. The real problem is making that happen. Cultural differences make it harder to sell to normal high-level management, so this is harder for companies selling to businesses than for something like WhatsApp. Most investors are from the dominant business leadership culture. Board membership is controlled by existing board members, and seats are traded as a currency of power. Avoiding those pitfalls is uncommon enough that some big existing company using 10x the people to get worse results can afford to buy out the owners, and founders are often tired and want to retire rich. Maybe you want to argue this is an intrinsic attribute of buyouts, but I've heard it's a lot better to sell your company to a Chinese SOE: they don't wreck your company culture nearly as much as American MBAs, they just want to transfer your technology.
Some people like to blame layers of hierarchy alone for such differences, with the idea being that there's a universal "management culture" that all companies are pulled towards. But that's just not true; companies of similar sizes and ages can have extremely different cultures. No, I think it's about pull towards a surrounding culture that sits in some local minimum. Again, I don't mean the overall culture of eg America, but specifically the culture of a management class.
Yeah, this isn't actionable intelligence for individuals. I wish I had something that was.
Yet, organizations can't function at all without some amount of technical expertise. How does that work in a politician-dominated organization? In my experience, it goes something like this:
me: If the current path continues, you'll run into [technical problem].
exec: I see, tell me more.
me: [explanation]
exec: (recognizing keywords) Ah, [field]. I have a guy for that. (smug)
note: This is not meant in the sense of "I have a doctor for that" or "I have a mechanic for that", which would be "we have a guy" instead. There was a sense of ownership over the person implied.
me: I suppose it can be considered [field]. I'd be happy to talk to that guy.
exec: Hmmm...should I schedule a meeting? His time is pretty valuable.
me: (More valuable than yours, huh?) I think this issue needs to be considered, but it's your call.
exec: (Considering the situation, I'm not sensing any political threats here.) OK then.
later...
me: I looked up your background then wrote this explanation to read to you.
expert: (misunderstanding)
me: (attempt at clarification referencing five other fields)
expert: I was brought in for my experience, and in my experience things were done this way. And now you're saying that needs to be changed?
me: As I explained earlier...actually, nevermind. You do your thing.
Loyalty of the expert here is considered as important as their expertise. I also think that in the minds of management, the more general a technical expert is, the more of a potential threat they are, and thus loyalty becomes even more important. If that's the case, it might be sometimes beneficial to pretend to be ignorant of areas outside your specialty, especially the business context of your technical work. (That's pretty inconvenient for someone like me.) Is that something you've seen?