To give a counter example: If one part of my body has been hijacked by a virus and is producing viral cells, I would want all the other parts of my body to fight back against that infection, rather than leave that part alone. Similarly when you talk about conflicts within organizations of societies, you should want people to take sides in a dispute between managers if one of them is embezzling or committing crimes.
So I think your model is beneficial for some forms of conflict but not others. You could perhaps draw a line for action at "things that threaten the viability of the whole setup" But the delineation of that would become its own controversy
In a well-founded marriage, spouses don’t try to induce internal conflict within their partner (e.g. shaming or guilting them) to win fights.
So I would expect that giving others a list of true information which connotes their relevant wrongness in some way on some topic (and may thereby induce guilt especially when the problem is explicitly stated) is not well-founded, according to you. Under well-founded environments, those with the advantage of existing unchallenged multi-prejudiced ideology would never be held accountable to their mistakes because all conscientious objectors can just be made into annoying squares.
Even worse, you contend that the opposite is to be "coherent" "like North Korea" "because everyone listens to the same person". So in your option model there's just no position corresponding to being virtuously willing to contend with guilt as a fair emergent consequence of hearing carefully considered and selected information.
Not sure what Richard would say, but off the cuff, I'd distinguish 'imparting information that happens to induce guilt' from 'guilting', based on intent to cooperatively inform vs. psychologically attack.
My read of the post is that some degree of "being virtuously willing to contend with guilt as a fair emergent consequence of hearing carefully considered and selected information" is required for being well-founded or part of a well-founded system (receive criticism without generating internal conflict, etc).
I'd distinguish 'imparting information that happens to induce guilt' from 'guilting', based on intent to cooperatively inform vs. psychologically attack.
Mhm, and in practice no one who accuses of guilt tripping actually cares about that distinction; if someone is being made to look bad then they basically never wonder if it's right. I'm not objecting to the 'guilt-tripping' framing for no reason; it's a thought-terminating cliche in 99.99% of cases where it's used.
[reading what I actually wrote here] ... And anyways 'inducing guilt' is what the most relevant informing-act looks like; if you're doing something wrong then you don't necessarily change it without attending to the exact details which would induce guilt. I never even said anything about 'guilting'; OP explicitly discouraged a correct thing to do without even mentioning 'guilting'.
There might be a third level to this approach. You can imagine that there are efficient vs inefficient coalitions. That is to say, some ways of organizing might be coherent (they do find courses of action), and well founded (have good recursive properties with regard to the coherence/properties of subagents under internal confict), but in which valuable trades do not happen, or overall overhead is high.
A good example is well vs badly managed companies. Even if there is not infighting, and they do come to some decisions, some companies do good jobs of actually achiving their goals given the individual competence of their members, and others have very competant subagents, who organize well and just structurally don't exectute.
So I think that you can measure the degree to which the agent is the most effectual organization of some subagents (for instance is task splitting efficient), especially past the human scale where coalitions are more freely formed.
Why is coherence a necessary base for well-foundedness?
Well founded marriage seems like something different from east and west halves of a country, there is a choice and pattern of behavior here, not a fundamental difference that stops it from being possible to create sub-agentic infighting. (East and West could pay for spies etc, but this isn't a fundamental part of the problem)
A truly stable system is a system that allows diversity, tolerates conflicts, and can guide the benign resolution of conflicts internally.
“If a kingdom is divided against itself, it cannot stand. And if a house is divided against itself, it cannot be maintained.” - Mark 3:24
In my last post I argued that we should view intelligent agents as coalitions of cooperating and competing subagents. The crucial question is then: how can we characterize effectively-functioning coalitional agents? One standard criterion is coherence: the extent to which the agent acts as if it has consistent goals and beliefs. In other words, an agent is coherent insofar as disagreeing subagents are able to still act as a functional coalition. For example:
Coherence is a valuable property for a coalition to have, but I think that characterizing idealized agents primarily in terms of coherence gives us an impoverished understanding of them. For example, a coalition which is highly-coherent because its leaders exercise a lot of top-down control is much less robust than a coalition which is highly-coherent because all its subagents actually want to cooperate with each other.
In this post I try to capture the difference between these two possibilities in terms of a property I call well-foundedness.[1] I define an agent as well-founded to the extent that conflicts between its subagents don’t propagate down to induce conflicts within those subagents. For example:
I think of well-foundedness as complementary to (and in some ways dual to) coherence. Ideal agents should have both properties. I don’t yet know how to define well-foundedness precisely, but in the rest of this post I characterize it informally by describing the four possible combinations of coherent/incoherent and well-founded/poorly-founded.
Consider two countries each experiencing strong internal political polarization, but in very different ways. In Holistan, the two factions are the one representing East Holistan and the one representing West Holistan. By contrast, in Fractistan the two factions represent two subpopulations who live closely intermingled throughout the country—say, two religious or ethnic groups.
For each country, increasing internal tensions makes them less coherent—each half of the country thinks of the other half as their enemy. But the two conflicts will play out very differently. In Holistan, you might see the East and the West start to cut ties with each other; set up parallel governance structures; or discourage travel or trade between them. If conflict continues to escalate, you might see a civil war, with each side drawing on their territory and population to muster an army.
This is pretty bad! But despite being very incoherent as a country, Holistan is still relatively well-founded, because each of East and West Holistan are internally still pretty coherent. What that means is that they have a line of retreat from conflict. The two sides in Holistan can still disengage; they can draw up peace treaties; they can form two separate countries. After the war ends, the fabric of each society remains intact—colleagues are still on amicable terms, neighbors still trust neighbors, city councils can still debate issues without relitigating the war with each conflict.
Perhaps the most vivid illustration of how important this is comes from World War 2. I am often struck by how, after the most devastating war the world has ever seen, European countries quickly recovered to unprecedented heights of prosperity, and even became close allies. I think a significant reason for that is that World War 2 was a relatively well-founded conflict between nation-states. It didn’t turn them into low-trust societies.
By contrast, Fractistan is neither coherent nor well-founded. The lack of clear territorial boundaries between the two factions makes it harder for rising tensions to erupt into a full-scale civil war. But the whole country is affected regardless: each region and city and neighborhood faces an internal power struggle. You might see lynch mobs or pogroms; or, in an extreme case, the kind of decentralized genocide that happened in Rwanda.
And there’s no easy way to end the conflict. Even if a nation-wide compromise is reached, each person will still be surrounded by former enemies. Each small-scale flare-up of renewed conflict will trigger further cycles of escalation. In other words, Fractistan will persist as a country, but with low social trust for the indefinite future. I picture it with a fracture running from top to bottom, fractally dividing the country in two at every level of organization.
Aside from Rwanda, two countries with Fractistan-like conflicts were Bosnia and Herzegovina and pre-partition India, which both saw widespread neighbor-on-neighbor violence. India was lucky to have a strong geographic separation between the bulk of its Hindus and Muslims. Even so, however, the process of splitting India and Pakistan was incredibly messy and cost hundreds of thousands of lives.
Coherence and well-foundedness are separate properties which are both individually valuable. But there are some tradeoffs between them. To explore those it’s useful to consider the opposite of Holistan: a country which is very coherent but also very poorly-founded.
I could make up an example here, but we already have one that matches the description very well: North Korea. As a country, it’s extremely coherent. Its whole government follows the instructions of one man. Its whole society follows the instructions of its government. There’s no national-level dissent, nor regional-level dissent, nor even dissent on the level of local communities.
But repressing dissent doesn’t make it go away—it just pushes it down to lower-level subagents. I expect that some North Koreans feel safe to dissent within their families; others only within the privacy of their own thoughts; and others not even there, but only in their subconscious shadows. Even when this dissent never surfaces openly, it is visible in the cost and scale of the control apparatus required to keep it repressed. If that control apparatus ever breaks, North Korea’s coherence would fall apart.
Such extreme repression is obviously bad. But in moderation repression can be a valuable driver of coherence. Henrich hypothesizes that the success of the West was driven by the Catholic Church’s prohibition against cousin marriage, which made Christian Europe less clannish. If true, you can think of this as trading off well-foundedness for coherence: by repressing kinship-based networks, the Church made larger-scale cooperation possible. More generally, societal morality works by repressing the antisocial instincts of each individual (as well as groups organized around antisocial behavior, like criminal gangs).
A similar set of tradeoffs arise in individual psychology: people can become more disciplined by repressing their emotions. This makes them more coherent—e.g. they can choose to work long hours on things that are instrumentally useful. But it often harms their ability to enjoy themselves and to understand and process their underlying motivations.
Conversely, to become well-founded, you need to surface ways in which conflicts manifest at low levels and then resolve them. This requires the opposite of repression: expression. Specifically, it requires that lower-level subagents are able to express their true preferences (as they can in individuals who freely let their emotions surface, or countries which let political dissidents speak freely).
What are the tradeoffs between building coherence via repression, and building well-foundedness via expression? I think of the former as making the average-case outcome worse, but also reducing variance. Repression is therefore appropriate when you’re in a scarce environment—one in which a big loss could totally wipe you out. An individual whose career could be ruined if they let their emotions show needs to repress them (even if it makes them more stressed and less productive on average). And a country which could be invaded if it gets distracted by internal politics needs to repress dissent (even if there’s something valuable to be learned from that dissent).
By contrast, expression tends to lead to better outcomes, but at the cost of also increasing variance. Expressing underlying conflicts allows them to be solved directly, but makes the overall agent less coherent until things actually resolve. For example, instead of sniping at each other about household chores, a married couple could express the emotional fears that underlie those frustrations. If that goes well, they’d feel much more respected and appreciated afterwards; but if it goes badly it could provoke a (potentially relationship-ending) fight. In a political context, letting dissidents speak out could lead to valuable reform, but it could also give rise to a full-fledged separatist movement.
So the tradeoff between repression and expression is a nuanced one, and needs to be decided on a case-by-case basis. But in general it’s likely that we err too far on the side of repression, because we don’t intuitively realize how abundant the world has become. Most countries don’t face significant risk of invasion, and therefore the costs of secession would often be outweighed by the benefits (more national cohesion, better governance, and being more well-founded in general). On an individual level, we can now move between different communities far more easily than at any previous point in history, and so taking the social risk of letting out our emotions is less dangerous than our intuitions are calibrated to expect. (There are some prominent exceptions, which I’ll discuss in a follow-up post, but I’m trying to keep this post relatively free of politically controversial examples.)
I think the tradeoff I’ve described above is an important dynamic in almost all real-world agents. But I don’t think it’s inevitable. We can imagine coalitions designed so that low-level agents can express their preferences, and make local improvements, without threatening the stability of the overall coalition.
What would such designs look like? I expect that a key component is “peace treaties” between high-level subagents, where they all agree not to use certain types of low-level conflicts to further their own ends. We can think of liberalism as a peace treaty which allows people with different religious and political beliefs to coexist. Meanwhile, arrangements like capitalism and democracy channel conflicts into formats that are productive (like business competition and political campaigning) rather than destructive (like theft or violence).
But well-foundedness at one level requires coherence at levels below that—otherwise it’s easy for conflicts to propagate downwards. And the systems I describe above aren’t very good at creating (or maintaining) lower-level coherence. In Reno’s terminology, they are “weak gods” whose primary purpose is to help different groups coexist, but which don’t have strong opinions about what those groups should actually care about. Conversely, “strong gods” like nationalism provide the substantive ideological content capable of unifying people under a single coherent identity, at the cost of excluding outsiders. The challenge of designing an idealized coalitional agent can be seen as the unsolved problem of balancing weak and strong gods across many different scales.
A well-founded set is one which has a bottom element. By somewhat tenuous analogy, a well-founded agent is one whose internal conflicts bottom out. Unlike the set theory definition, my notion of well-foundedness is a matter of degree: the further down an agent’s internal conflicts go, the less well-founded it is.