Thanks for the references! I now know that I'm interested specifically in cooperative game theory, and I see that Shoham & Leyton-Brown has a chapter on "coalitional game theory", so I'll take a look.
If you have two strategy pairs (x0,y0),(x1,y1), you can form a convex combination of them like this: Flip a weighted coin; play strategy 0 on heads and strategy 1 on tails. This scheme requires both players to see the same coin flip.
A proof of the lemma V(a,b∗)≤v∗:
Ah, ok. When you said "obedience" I imagined too little agency — an agent that wouldn't stop to ask clarifying questions. But I think we're on the same page regarding the flavor of the objective.
Might not intent alignment (doing what a human wants it to do, being helpful) be a better target than obedience (doing what a human told it to do)?
Also Dan Luu's essay 95%-ile isn't that good, where he claims that even 95th-percentile Overwatch players routinely make silly mistakes, suggesting that you can get to that level by not making mistakes.
Oh, this is quite interesting! Have you thought about how to make it work with mixed strategies?
I also found your paper about the Kripke semantics of PTE. I'll want to give this one a careful read.
You might be interested in: Robust Cooperation in the Prisoner's Dilemma (Barasz et al. 2014), which kind of extends Tennenholtz's program equilibrium.
Ah, thank you! I have now read the post, and I didn't find it hazardous either.
More info on the content or severity of the neuropsychological and evocation infohazards would be welcome. (The WWI warning is helpful; I didn't see that the first time.)
Examples of specific evocation hazards:
Examples of specific neuropsychological hazards:
I know which of these hazards I'm especially susceptible to and which I'm not.
I appreciate that Hivewired thought to put these warnings in. But I'm kind of astounded that enough readers plowed through the warnings and read the post (with the expectation that they would be harmed thereby?) to cause it to be promoted.
Oh I see, the Pareto frontier doesn't have to be convex because there isn't a shared random signal that the players can use to coordinate. Thanks!