It seems like for embedded (reflexive, Löbian, etc) LDT agents, there ought to be a fixed point thing between decision and prediction.
Indeed, embedded agents can predict things about their own actions; but by modeling themselves sufficiently well, this should be (in the limit) equivalent to making a decision, as they will be modeling their own thoughts. Conversely, once you have decided, if you do not suffer from akrasia, then you have accurately predicted your next action. (aside: this is the source of the illusion of free will.)
This is related to the class of "metaphysical truths": truths of the form . Whenever an embedded agent believes one of those, then it must (by Löb's theorem) eventually believe . But there are lots of such truths (perhaps each different religion offers a different set of metaphysical truths), which might then lead to spurious, or even contradictory beliefs!
The key word was "eventually", assuming LDT agents are logical inductors of some kind; in the meantime, the agent may choose its beliefs. Isn't this weird? Beliefs shouldn't be arbitrary!
But you can imagine, as an (imperfect) example, the paradox of self-confidence: if you think you are competent, then you could believe in your ability to self-improve, which will encourage your to foster your own competence; on the other hand thinking that you are incompetent may lead to not believing in your self-improvement ability, leading to a downward spiral.
Each one of these are decision-belief fixed points. Each are, in way (causally?), both true and rational.
I feel like LDT will end up being a reflexive fixed point of this sort (reminiscent of the logical induction fixed point), with the catch that there are many such fixed points. The true decision an LDT agent must make is then choosing the most effective of these fixed points.
(I'm not entirely convined of this yet since I still have no idea what logical counterfactuals will look like)
The moral of the story for us humans is that:
- akrasia should not exist, not if you can predict yourself well enough;
- sometimes beliefs are arbitrary. choose the most productive ones, you'll end up believing them all anyway.
EDIT: I feel this is rather important, so I would appreciate getting feedback on the presentation.