My actual literal nightmares about civilizational collapse somehow manage to be insanely optimistic about human nature.

I dreamt that in response to the news of the Trumps’ probable successful intimidation or bribery of their New York prosecutors, the US devolved into a lawless hellscape, since the last shreds of pretense of “we’re punishing you because it’s what the law says” were gone. In my dream, I successively wished I’d transferred more of my assets to paper, then money, then gold, then firearms, as I realized how far things had gone.

If I’d been thinking sanely, the thing I should have wished I’d accumulated is the only real source of safety in a state of war: a bigger, better gang. But fundamentally, I should have known better than to imagine that things would collapse quickly.

What was I getting wrong? I was tacitly assuming that the majority of people were perfectly principled.

The rule of law and the structure of power

The crazy thing about my nightmare was that it assumed that we had rule of law in the first place, that most of the sorts of people who vote for “law and order” politicians meant the same sort of thing I do by law. In practice, at least for decades, they’ve meant whatever “we” can get the police to enforce, to preserve a civil order that keeps the right sort of people on top.

The normal attitude towards police officers, soldiers, and other authority figures entitled to use coercive force, is to feel safe if they seem like they’re on your side, and unsafe if they seem like they’re the enemy. Police are presumed to follow the laws insofar as there is a custom to do so, and they feel that the power structure they’re embedded in wants them to; exceptions are judged on the basis of whether they locally cause harm.

But my intuitions about whether I am protected by police are quite different. The motive I imagine for following legal procedure is a transcendent commitment to following the law in full formality, because it is the law laid down in the civil code.

This explains how other people - especially other people in groups comparatively unlikely to be the targets of state violence - can read stories about cops and courts colluding to screw over some group or other and maybe feel vaguely guilty, but not personally unsafe. My intuitions are basically always that if we have a good shared set of protocols / laws, then it doesn’t matter whether we’re friends, we’ll do the right thing anyway, and if we don’t, then it’s hopeless. But these intuitions are not terribly common, and they’re not a good match for how rules are actually enforced in our society.

In particular, I’ve systematically underestimated how OK things can look to a privileged person without proper rule of law, bc predatory structures still want to allow production that can be taxed.

Law as command structure

If you’re a predatory institution or other extractive coalition, you will often want to be able to coordinate local action. To do so, you’ll need some sort of standard data structure and corresponding behavior schema. If you’re not fastidious about making statements that are literally universally true, you’ll often end up making universal statements that are locally true. In addition, if you’re trying to do this efficiently, you’ll often pick up schemas someone else left lying around, such as legal frameworks invented by systematic thinkers who were trying to be consistent. But since you’re trying to make local decisions, not report on your structural model of reality, you won’t be overly distressed by the fact that sometimes you just know that the rules don’t apply, without a formal articulation. All that matters is that you’re consistent where that is important to the performance of your coalition.

This resembles Wittgenstein’s idea of a command language, which can easily superficially resemble pure structural language. If you can get the people who believe in true rule of law to join your coalition under the misapprehension that that’s what you’re doing, so much the better.

Verbal consistency as submission to official narratives

You could draw a quadrant diagram with 2 axes. One describes the public account people give of their actions - the extent to which they try to fit into an intelligible system (with corresponding defensibility), or explicitly endorse acting based on feeling (with corresponding unaccountability). Call these Ordered and Subjective, respectively. The other describes how formal reasoning and models fit into their internal decisionmaking process - whether they have a variety of unintegrated ad hoc models, or a single, integrated model. Call these Opportunistic and Autonomous.

When I say that I was tacitly assuming everyone was perfectly principled, I mean that I was assuming that everyone was either Ordered Autonomous or Subjective Opportunistic. Basically I was assuming that all the people talking about law and order, police/courts ever not abusing their power Because Of The Law, etc are in Structured Autonomous, following the law as written because they are personally committed to following the law as written. That they have internalized the law so that they follow it autonomously. This outlook is basically Kantian.

But in fact they are Ordered Opportunistic, trying to work within the context of power structures, and taking refuge in their ordered system when challenged, but not acting based on an internalization of the full formal structure. They engage with the words and procedures because those are part of the apparatus of power dynamics, part of how things are done here, which means that they're locally consistent, but they don’t report their true underlying model of what's going on, and maybe don’t have one. If the power dynamics shift, their sense of what is lawful shifts accordingly, to adapt to the new ruling coalition. Law is just a way coalitions work together to hold onto power. Call these Collaborators.

How does this relate to the desire to pass tests?

Most people in developed countries spent much of their formative years in contexts where they were explicitly evaluated, by authority figures whom they could not physically escape, on the basis of their ability to verbalize officially endorsed narratives. For people with an autonomous disposition, this can lead to fully internalized domination; in order to recite the desired phrases, they need to alter their sole internal model of how things are, to match. But for people with an opportunistic disposition, they can simply spin up a sandboxed narrative to recite the teacher’s password, and drop the narrative when the test is over.

This helps explain part of why hedgehogs so often make poor predictors. One commonly discussed result of Philip Tetlock’s research on forecasting skill is that “hedgehogs” - people who reason based on a single integrated model - underperform “foxes”, who mix together multiple considerations.

Some people who verbally report a single explicit model have an autonomous disposition. But they’ve spent a long time in situations in which some narratives are favored for reasons other than truth-tracking. Since these narratives are not sandboxed, they have to be reconciled with the rest of their narratives, achieving consistency at the expense of predictive accuracy and explanatory power.

For opportunists reporting a single model, the situation may be even worse. Their single explicit narrative is not constrained by their need to function in the world, since they make pragmatic decisions using a different mental module entirely. So their reported beliefs track a convenient consistent worldview, but they don’t use the vast majority of their practical knowledge and life experience, and can’t change their mind when it’s not socially convenient to do so.

Defensive irony

So, what is an autonome to do?

One solution is to speak the truth despite social incentives to do otherwise. The problem with that approach is that social incentives are powerful. Most likely you end up deluded anyway - but also socially unsuccessful. However, this approach might be able to work, if you take into account the way in which you’re handicapping yourself, and take active steps to overcome this handicap. For instance, you might form a truth-seeking community with explicit understanding that you’re not seeking the approval of outsiders, and willing to forfeit the goods that would come with such approval.

Another is to simply lie. To keep two sets of narrative books - a public one and a personal one. This is cognitively expensive, but it makes it easier to see the pressures your external narrative is under. Another problem with this strategy, of course, is that you’re collaborating with the pressure on other people to conform to the official narrative, and you lose out on the chance to locate potential intellectual partners.

A third strategy is irony. The Socrates of the Platonic dialogues clearly had a specific, structured critique of Athenian discourse - but often fell back on the claim that he knew nothing. That his only wisdom was knowing how little he knew. This is the famous "Socratic irony." Nietzsche proposes a related sort of irony, in which he says that the truly creative person will often want to hide, dissemble, wear masks, that innovators will claim to merely be old-fashioned. More recently, I talked with someone who claimed that her moral philosophy was “fuzzy utilitarian” - but when I probed for details, it turned out that her problem with most attempted implementations of utilitarianism was that they themselves were too “fuzzy”. Her self-deprecating description as “fuzzy” secretly meant that she thought her thinking was more precise.

Irony occupies the Subjective Autonomous quadrant. Internally, you keep accounts. Externally, you disavow claims to lawfulness, so as to avoid attaching yourself to structures of predation.

Protean

Then there's the last quadrant, Subjective Opportunistic. This is the other honest quadrant. People following this strategy distrust external systems of accounting sufficiently to simply avoid any accounts, as they (correctly) see most lawful systems as power grabs. Unlike the Ordered Opportunistic, they don't try to claim credit for being part of a lawful system - the strategy is defensive. Shapeshifters are hard to grab a hold of.

Principles and education

Going back to my nightmare, it seems like it’s not a coincidence that Ayn Rand has the same unrealistic assumption in her fiction that people are perfectly principled, and also tacitly expected a rapid civilizational collapse, as shown by her fiction.

(Her idea of perfectly selfish people make herculean efforts to hold up their end of contracts regardless of incentives. Her perfectly selfish business owners are happy to be competed out of business by someone with a better product, and freely share their trade secrets. Her perfectly selfish philosopher accepts a humble life as a burger-flipper instead of accepting prestige from a corrupt system and thereby legitimating it. This is … not most people’s idea of perfectly selfish behavior.)

It’s also probably not a coincidence that I grew up Jewish. And that Ayn Rand also had a Jewish background. (Or that the US Supreme Court is entirely staffed by Jews and Catholics). I hadn’t noticed just how weird it is that I know lots of people - liberal atheists in liberal atheist communities - who observe some ritual laws specified thousands of years ago, despite no social pressure to do so, that don’t even reinforce current power imbalances, just because it feels right to connect with tradition.

Reverence for the rules is not totally unique, of course - Germans are famous for “but the rules......,” and Scandinavians seem like they have a similar thing going on. But German-Americans and Scandinavian-Americans don’t seem to have the thing.

But I suspect that something about growing up in a context where every Saturday we brought out a holy scroll with a story containing a law code, paraded it around the room, read from it, and sang songs about how it’s a special gift given to us by god, king of the world, it is the tree of life and all its paths are pleasant, and everyone who holds onto it is made happy, might have had something to do with my attitude towards law. And that not everyone had this experience.

And my exposure to a mixing of ritual law (including things like food taboos) and moral law (including things like charity and fair dealing) may have something to do with how I intuitively expect judges to uphold the law, not because they have a social incentive to do so, not because it accords with their self-image, not because it’s their custom and training, but because it would be unclean to do otherwise.

More precisely, the wrong assumption I made was the assumption that it is common knowledge that everybody is perfectly principled. If I aware of the existence of contingent civil order as a byproduct of power structures, I might expect it to continue for a while, and behave accordingly, even if the judges are unclean.

Engineering and the transcendent

Engineering used to seem impossibly hard to me. Permaculture did not feel impossibly hard. Neither did the tinker trade.

What’s going on?

Basically I took engineer superhero / Robinson Crusoe style stories at face value as honest accounts of what engineers can do.

In practice, engineers do what they to by relating to systems with finite bounded scope & affordances, and copy known techniques. I was assuming they had a true art (techne) in the Platonic sense. It seems impossibly hard to build a modern car or computer from first principles, since that implies knowing how to build a bronze age level civilization with many types of mining and trade.

But of course engineers use the tools they find, including know-how copied from books.

Reading The Martian recently helped me recalibrate, because it did not seem impossible, just hard. But, obviously, the engineer hero in The Martian was not creating a full-stack civilization on Mars. He was completely dependent on a static endowment of tools and materials, the technical skills he already had, and the things he could look up.

Engineers are pragmatists, like tinkers except that they can trade with an industrial base, and have access to more books. They basically don't do anything from first principles, and don't actually do whole-system maintenance. So, when I it seemed impossibly hard to do really know how to do e.g. computer programming or bridge building from scratch, the answer is that they just don’t. No one does.

And that’s OK.

But in some domains, I think we can do better. There’s a reason why the culture that produced an outsized number of science Nobelists is not an engineering culture, but a rule of law one.

Related: Geometers, Scribes, and the structure of intelligence, Be secretly wrong