I've written up a rationality game which we played several times at our local LW chapter and had a lot of fun with. The idea is to put Aumann's agreement theorem into practice as a multi-player calibration game, in which players react to the probabilities which other players give (each holding some privileged evidence). If you get very involved, this implies reasoning not only about how well your friends are calibrated, but also how much your friends trust each other's calibration, and how much they trust each other's trust in each other.

You'll need a set of trivia questions to play. We used these

The write-up includes a helpful scoring table which we have not play-tested yet. We did a plain Bayes loss rather than an adjusted Bayes loss when we played, and calculated things on our phone calculators. This version should feel a lot better, because the numbers are easier to interpret and you get your score right away rather than calculating at the end.

10 comments, sorted by Highlighting new comments since Today at 8:24 AM
New Comment

For those interested in running the game, I put together a Python script that pulls questions from an API for trivia questions, and outputs 2 LaTeX files that can be compiled and used, one for the quizmaster, plus a set of corresponding cut-out cards for the players.


Pull requests and improvement suggestions are welcome!

I wonder if the games you played had resembled the expected Aumann process, which is akin to random walk, or did they look more like slow convergence of opinions? If it's the latter, then the game has little to do with Aumann agreement.

Regardless of how well it follows the random walk, it already violates the assumption of rational agents.

Then why take Aumann's name in vain?

I think the relationship to Aumann's theorem is direct and strong. It's the same old question of how Aumann-like reasoning plays out in practice, for only partially rational agents, that was much discussed back in the Overcoming Bias days.

May I suggest that "Aumann Agreement Game" would be a better name than "Aumann's Agreement Game" because the latter suggests (falsely, I take it) that the game itself is Aumann's?

[EDITED to add:] In case it's not obvious, the title and content of this post used to say "Aumann's" rather than "Aumann".

Yes, good point.

I like this very much. Did the game work in practice as you describe in the example?

Essentially, yes! There were often a few more revisions than this, and the trolling was more subtle.