My best guess is that the sort of strategy hinted at here can scale up to a hundred people, but not much farther. It's relatively common wisdom that a small startup can outperform a big company by being more nimble and coordinated. But, small startups still eventually (often) scale up into big companies – for the simple reason that there's still a limit to the power and profitability a small startup can achieve.
This brings to mind a somewhat different lens into the institutional failure modes here, that a friend described to me recently. (I'm not 100% sure I grok or agree with this view, but will try to explain my version of it here)
tl;dr: Good Organizations scale to the point where competent people are spread thin juuuuust to the point where werewolves can rise in power, without affecting the innermost circle of power.
Organizations that scale less than that are outcompeted by those that do. Organizations that scale more than that self destruct.
...
The post How to Use Bureaucracies notes:
A bureaucracy is an automated system of people created to accomplish a goal. It’s a mech suit composed of people. The owner of a bureaucracy, if an owner exists, is the person who can effectively shape the bureaucracy. Bureaucrats are people who are part of a bureaucracy (excluding the owner).
Not all organizations are bureaucracies. Most organizations are mixed — they have both bureaucratic and non-bureaucratic elements.
The Purpose of Bureaucracies
The purpose of a bureaucracy is to save the time of a competent person. Put another way: to save time, some competent people will create a system that is meant to do exactly what they want — nothing more and nothing less. In particular, it’s necessary to create a bureaucracy when you are both (a) trying to do something that you do not have the capacity to do on your own, and (b) unable to find a competent, aligned person to handle the project for you. Bureaucracies ameliorate the problem of talent and alignment scarcity.
I think this can often generalize somewhat to non-bureaucracy institutions – a competent person (say, the founder(s) of a startup), will eventually want to scale their org so it can accomplish more things. One of the ways to scale it involve lots of bureaucracy. Others involve distributed teams that mostly pursue their own agenda while contributing to a common good, sometimes through non-bureaucratic hierarchy. (Where there's a series of managers. At each ring of the hierarchy, managers ostensibly coordinate on high level projects. But the environment that allows promotion within the hierarchy creates incentives for werewolves)
Scaling the company inevitably produces wasted motion, in some form. And you might look at most of the stuff happening, and think "man, this is so dystopian and sad. Can't we just coordinate better?" and the answer is yes, you can, but not at this scale.
Somewhere at the top of the company is a small team of founders, who (maybe) actually trust each other, have good plans, preserve absolute power, and carefully vet the people they allow into the innermost circle. They have worldscale goals, so they have scaled the company to function on the worldscale. They know that this means, underneath them, this has Games of Thrones and tons of wasted bureaucratic motion. But this is a necessary byproduct of extending their own power.
Biological metaphor
In biological evolution, there's some reason to think humans are the stupidest example of a smart-person-architecture you could come up with – as soon as evolution produced something smart that could reasonable scale, it straightforwardly scaled it without spending a lot of time fine tuning the architecture.
When humans have goals they can't accomplish on their own, they build orgs. When the orgs aren't big enough, they expand the orgs. A lot of human goals are "be able to compete on the worldscale, where at least some people are competing infinitely – they want to be the best, most profitable company or country. Because at least some orgs have unbounded goals, even if your goal is bounded, if it impacts the worldscale you must contended with cancerous, unbounded agents.
This means every competitive org must be the largest, most rickety versions of themselves that can reasonably function.
If there was an org that figured out how to be less rickety, with less wasted motion... they would spend free energy on expansion, gaining more resources, turning those resources into more actions.
Kingdom metaphor
If you're a king with 5 districts, and you have 20 competent managers who trust each other... one thing you can do is assign 4 competent managers to each fortress, to ensure the fortress has redundancy and resilience and to handle all of its business without any backstabbing or relying on inflexible bureaucracies. But another thing you can do is send 10 (or 15!) of the managers to conquer and reign over *another* 5 (or 15!) districts.
Each district produces grain, and more population. At least some of the population is... at least a mediocre or moderately good manager. A given district performs better if it has a competent manager running the fields, the schools, the guilds, and the military. But the kingdom as a whole performs better if each district has one competent manager, and relies on mediocre managers for each of its subtasks.
Mediocre managers are not skilled enough to use very good game theory – they're just skilled enough to add friction to the game of thrones, to keep it sluggish and predictable enough that the Competent Managers can see what's coming and step in when it's about to get out of hand.
i.e. werewolves get to flourish up till the point where they could affect the Inner Council of the King. By then, they have revealed themselves, and the Inner Council of the King beheads them.
This is bad if you're one of the millions of people who live in the kingdom, who have to contend with werewolves.
It's an acceptable price to pay if you're actually the king. Because if you didn't pay the price, you'd be outcompeted by an empire who did. And meanwhile it doesn't actually really affect your plans that much.
Ramifications / Takeaways
This is all described in oversimplified and idealized terms. I think in the real world, there often isn't an inner circle of competent people who detect werewolves before it's too late. And I do think you can outcompete other orgs by actually having an inner circle of people with high trust. And figuring out tools to expand that circle and scale their coordination is quite an important goal – I have reason to think there's low (or mid) hanging fruit here.
But I do currently think that at the end of the day, this doesn't result in a world that looks all that different. Behind the veil of anthropics, you should mostly expect to be located in one of the kingdoms sacrificed to the werewolves. And as you explore different kingdoms and hierarchies, you should mostly find more of the same. And this would be the case even in a world where the leaders of orgs are mostly good people (but at least some have opposing unbounded goals)
There is also overhead to scaling and difficulty aligning goals that they want to avoid. (As above, I think my Ribbonfarm post makes this clear.) Once you get bigger, the only way to ensure alignment is to monitor - trust, but verify. And verification is a large part of why management is so costly - it takes time away from actually doing work, it is pure overhead for the manager, and even then, it's not foolproof.
When you're small, on the other hand, high-trust is almost unnecessary, because the entire org is legible, and you can see that everyone is (or isn't) buying in to the goals. In typical startups, they are also well aligned because they all have similar levels of payoff if things go really well.
On Twitter, Freyja wrote:
This intersected well enough with my current interests and those of the people I've been discoursing with most closely that I figured I'd try my hand at a quick explanation of what we're doing, which I've lightly edited into blog post form below. This is only a loose sketch, I think it does reasonably precisely outline the argument, but many readers may find that there are substantial inferential leaps. Questions in the comments are strongly encouraged.
Any serious attempt at (b) will first have to unwind the disinformation that claims that the thing we have now is capitalism, or remotely efficient.
The short version of the project: learning to talk honestly within a small group about how power works, both systemically and as it applies to us, without trying to hold onto information asymmetries. (There's pervasive temptation to withhold political information as part of a zero-sum privilege game, like Plato's philosopher-kings.)
Some background: post-WWII elite institutions (e.g. corps) are competitive to enter, but not under performance pressure, because of US government policy. This strongly selects for zero-sum games, which mimic but wreck discourse. (See Moral Mazes for more, especially the case studies that make up most of the book, starting around chapter 3.)
This creates opportunity in two ways.
First, institutions are mostly too stupid to model their environment beyond the zero-sum games they specialize in, so a small group that's able to maintain information hygiene and not turn on each other should be able to take & hold territory. "And not turn on each other" turns out to be really hard, because all our role models and intuitions for how to survive in this world involve doing that all the time. But we're learning!
(A mundane example of a decisive advantage due to information hygiene: Paul Graham writes about how his startup did better because it used an elegant programming language. That's only information hygiene on the purely technical level, but that was enough to outmaneuver huge corporations with a strong perceived incentive to ruin them, for quite a while. For a less mundane example, the story of how Elisha outmaneuvered multiple ruling dynasties is a personal favorite - 2 Kings 5-10. The narrative distorts the "miracles" a bit but it's not hard to reconstruct how he actually did it.)
Second, because most supposed productive activity is done in the context of huge stable corporations, people are trying to maximize the number of jobs and complexity per unit of output. This implies that many things can be done much more easily.
So that implies that if we can have good enough information hygiene and group cohesion not to fall victim to the perverse impulse to do the kind of make-work or artificial scarcity that creates much of cost disease, we can learn how to build a nearly full-stack civilization in a small city-state. Obviously there are many steps between here and there, but since lots of them involve getting collectively smarter, a detailed plan would be inappropriate.
What does good information hygiene and group cohesion look like? The game Werewolf is a good example. Players are secretly assigned the identity of Villager (initially the majority) or Werewolf (minority). Each round all players vote one player out, and Werewolves secretly do the same. There are other details that allow villagers to make some inferences about who the werewolves are. But they have to play the first few rounds right or they lose.
Optimal play for Werewolves involves (a) targeting whichever villagers are the most helpful to public deliberation, for exclusion, and (b) during public deliberation, being as unhelpful as they can get away with while appearing to try to help at other times. I realized a lot of things about how social skills feel from the inside when I finally figured out how to play correctly as a Werewolf.
Optimal play for Villagers involves creating as much clarity as possible, as soon as possible, and being willing to assume that people who seem to be foolishly gumming up the works are Werewolves if there's no other clear target.
With optimal play, Villagers usually win, but in practice, at best one or two people try to create clarity and are picked off in the first round by the Werewolves. The other Villagers are resigned to trying to die last, so they lose.
The thing I said about elite culture favoring zero-sum games can be recast as: the social environment favors playing Werewolf over playing Villager. In case it's not obvious, optimal real-world play for Villagers can often involve leaving the Werewolves alone. In real life there are better things to do than murder your enemies, like hang out. Villagers just need to defend themselves if and when they're actually threatened.
We're trying to learn how to play the Villager strategy successfully, in a context where we've mostly been acculturated to play as Werewolves, especially among elites. This has to involve figuring out how to do interpersonal fault analysis (identify when people are being Werewolfy) without scapegoating (assuming that fault -> blame -> exclusion).
In other words, justice seeks truth, but intends to leave no one behind; people who can't contribute need to feel safe admitting that, and people who hurt the group need the option to repent & heal the breach.
We don't have great finesse yet but optimal play in our world seems to be some fluid integration of talking about politics, healing personal trauma, and intersubjective openness.
Havel's The Power of the Powerless describes a similar (but less self-aware) strategy which he calls "dissidence." He (accurately, I think) predicts that the situation in Capitalist countries will be more difficult than the situation in Communist ones, because Capitalist ideology is more persuasive because it's more plausibly true.