For this Sunday's LessWrong meetup, I'll be giving a short presentation on some moral/game-theoretic/coordination problems and solutions I've been thinking about, which I would like feedback on.
Three related puzzles:
- Sometimes, people disagree on coordination protocols, or on moral frameworks. What do we do about that in the general case?
- Many rationalists explore novel coordination principles. This can result in us learning about "new and exciting ways that everyone is defecting on each other all the time". If a principle is novel, it's hard to coordinate around. How do we handle disagreements about that?
- Sometimes, Alice and Bob disagree on how to coordinate, but Bob thinks Alice has overall demonstrated better judgment, and it's (relatively) easy for Bob to defer to Alice. But other times, Alice and Bob don't trust each other, and each of them thinks the other is somewhat less sophisticated (despite having overall similar worldviews). How should they handle that situation?
"The Coordination Frontier" is my (placeholder) term for "the parts of morality/game-theory/coordination that aren't obvious, especially new principles that are novel advances on the current state-of-the-art." I think it's a useful concept for us to collectively have as we navigate complex new domains in the coming years.
I have some existing thoughts that I'd like feedback on, and I'd generally like to spark discussion about this topic.
Approximate format will be:
- I give a short(ish) presentation
- General Q&A and public discussion
- Brainstorming in a google doc, and then splitting into smaller groups to discuss particular subtopics.
- Return to a centralized conversation and share insights.
This will take place in the Walled Garden, in the Manor (to the north of the central map)
http://garden.lesswrong.com?code=MNlp&event=the-coordination-frontier
Note: Today we'll be meeting in Bayes Lecture Hall (across the hall from Bacon Lecture Hall, where we met the last couple meetups)
Here are the notes from the conversation.
https://docs.google.com/document/d/11IuD2k0R-KF5n-byg_O9WtIo1cak25lL3g-WBos417w/edit#
I'm copy-pasting them here as LW comments to make them a bit more visible
Marcin's notes:
My own original notes as I prepared for the talk:
Thesis:
It is possible to learn new coordination principles, with implications on how to optimally interact.
In an ideal world, you'd be able to state the principle and have people go "oh, that seems right", and then immediately coordinate using it. Unfortunately this is often not so easy.
If people don't understand your principles, you won't be able to use them to cooperate with people. (People might be able to make simplified guesses about your principles, but may either overfit or overgeneralize their response and not be able to respond in nuanced ways. They may also just decide you aren't worth interacting with as much)
It's harder to bring up new principles during conflict or high-stakes negotiations, because everyone knows it's possible to use clever arguments to persuade people of things falsely. I know I sometimes pursue "biased fairness", where there might be multiple fair-ish-sounding solutions to a conflict, but I'm incentivized to notice and argue for the one that benefits me. I worry that other people are doing the same. During a conflict, I neither trust myself nor trust other people to be as fair, clear thinking or impartial as they would be Not-During-A-Conflict.
During _stressful_ conflict, where people are operating in scarcity mindset, I trust them (and myself) even less.
People also just sometimes impose norms on each other in a bullying way that doesn't respect each other at all. The internet is full of people doing this, so people have defense mechanisms against it. I claim this is correct of people.
Thus, if you want to get a frontier principle or norm into the general coordination-toolkit for your community, I recommend:
1. Try to write a public blogpost *before* a major conflict comes up, where people have the ability to think clearly, argue, and mull it over *before* any high stakes implications come up.
2. In some cases, it might still be important to unilaterally enforce a norm or defend a boundary that people don't understand, or disagree with.
If Alice decides to unilaterally enforce a norm, my suggested "good sportsmanship" rules for doing so are...
(see: "Goodwill Kickstarter")
I'd summarize all this as "Alice can spend social capital on unilaterally enforcing a norm, or commanding people's attention to think more about the norm even if they don't think it makes sense on the face of it. This social capital is limited. Alice can also _gamble_ social capital, where if people end up thinking 'oh, Alice was right to enforce that norm', then Alice gains more capital than she loses, and gets to do it again later."
But, importantly, social capital isn't infinite. If you spend too much social capital, you might find people being less
I think this all goes more smoothly if it's actually an agreed upon meta-norm than if people are doing it randomly, and if it's made explicit rather than implicit.
One key problem: "social capital" is a vague abstraction that isn't super clearly tracked.
You also kinda have a different social bank account with different people who care about different things. I think humans are moderately good at tracking this implicitly, but not _that_ good. You might accidentally overspend your bank accounts.
People might also disagree about how much social capital you have to spend on a thing. If Alice unilaterally imposes a norm on Bob and Charlie, Bob might think this was okay, but Charlie doesn't. And then that can cause conflict, imposing costs on all three people.
There's a fine line between "judiciously spending social capital on things that are important" and "just bullying people into getting your way a lot, and maybe being charismatic enough to get away with it."
Gentzel's links:
Abram's notes:
John’s thoughts:
Doe's notes:
Some random additional concepts that feel relevant to me:
Goodwill Kickstarter – often, my willingness to extend someone goodwill depends on them being willing to extend me goodwill.
Humans have some default cognitive/emotional machinery for how to coordinate. At least sometimes, this machinery sort of assumes that other people have similarly to you.
Anger – anger credibly signals to someone that you might punish them, and that you might punish them again if they do the same action again.
Frustration – frustration signals to yourself that there is something suboptimal about a situation, which you have the power to fix.
Grieving – a process by which you come to terms with the fact there is something wrong that you *can't* change.
Sadness (the kind you tell people about), and being scared (the kind you tell people about) – tools for getting other people to help you (sometimes, helping you with the specific thing you're sad about or afraid of. Othertimes, simply reassuring you that you are understood, and valued, which signals that you can get help with other problems in the future)
Paralytic Sadness or Fear – tools for conserving energy or avoiding conflict.