Zac Hatfield Dodds's Shortform

1 comments, sorted by Highlighting new comments since Today at 11:04 PM
New Comment

Fully-general precommitment and its discontents

A cute trick for Newcomblike problems: precommit to "always act as if I had made the most advantageous precommitment".  This is the essence of functional decision theory; but operationalisation remains challenging.  Consider a Stag Hunt among FDT agents:

  • Without common knowledge (that each agent knows the others use FDT), FDT has nothing to say about the game - just calculate expected value by multiplying payoffs by probabilities as usual.
  • With common knowledge, FDT agents hunt stag, hurray!  But if we assume that we have common knowledge, did FDT really help?

I think it does, because establishing common knowledge of FDT among a population of agents requires less information than doing so for particular strategies.  Communication makes this easier, but isn't required:

Imagine a toy environment with an evolving population of {C,E,F}DT agents randomly assigned to various games (Newcomb's, smoking lesion, prisoner's dillema, stag hunt, etc.)each round, such that correctly believing that all or most agents are FDT is advantageous.  After some large number of rounds, surviving FDT agents may conclude that the distribution of games they have been assigned is representative of the environment; and therefore what fraction of surviving agents use FDT (handwaving some prior distribution).  We then have a meta-game of "at what round should FDT agents reasoning from random observations from a shared distribution switch to assuming that they are playing with decision-twins", which is exactly the kind of decision problem FDT is good for!

For more realistic environments, I think FDT pushes towards Governing the Commons -style mechanism and institution design - informed by decision theory, but carried on with sufficiently different methods that it constitutes a different field.