An argument of Parfit's reconsidered with logical decision theory

transhumanist_atom_understander

A logical decision theory recommends that you choose as if deciding the output of your decision algorithm.

The main difficulty in formulating a logical decision theory is how to define statements like: "If my algorithm outputs this, the result will be that". These look like counterfactual implications, but counterfactuals describe possible worlds with different contingent facts. What our decision algorithm outputs (with given input) is a logical fact, and it is not clear what it means to say what would happen if a logical fact were different. Much theoretical work has gone into elaborating this kind of counterpossible implication, without, in my opinion, satisfactory results.

But what's the point of this theoretical work? Do we really need LDT?

What's the difference between choosing the output of your decision algorithm and just making a choice? There is no difference, as long as your choice is the only thing in the world that depends on your decision algorithm. The only substantial applications of LDT are games where the player's environment contains some other implementation of the algorithm. The central example is Newcomb's problem.

But Newcomb's problem sounds like a science fiction story, and I'm not convinced by the case that analogous problems are part of our ordinary social lives.

But leaving aside the prospect of applying LDT to real life, I do think it's important in philosophy. In this post I'm going to share an example that came up when I was reading Derek Parfit's Reasons and Persons. It's an argument that Parfit makes where LDT seems conspicuously missing. Like, if you were familiar with LDT, you would have gone in a different direction.

Derek Parfit's book introduces a zoo of theories of morality and rationality, which he refers to by single-letter acronyms. In the first part of the book (all I've read to be honest), he describes how these theories can be self-defeating, introducing a taxonomy of the varieties of self-defeat.

The basic principle of LDT becomes applicable when Parfit introduces "collective self-defeat". This variety of self-defeat depends on the outcome of a game in which multiple players decide according to the same theory. Because you know the other players are deriving their decisions from the same theory as you are, a philosophical move inspired by LDT yields interesting results.

Parfit's claim: S is collectively self-defeating

I'm going to use a single argument of Parfit's as an example. The subject of the argument is the self-interest theory (), which he says is a theory of rationality. (Since this is LessWrong, I must add that he doesn't mean what Yudkowsky means by "rationality", but I don't think that's important for this argument.)

What we need to know about is that it prescribes an aim, which is that your life go as well as possible, as well as what I'll call a policy: it recommends taking the actions that make your life go as well as possible, among the available alternatives.

Parfit argues that has the property of being collectively self-defeating: that is, if we all follow , then we will do worse by our -given aim than if we had all taken other actions.

(Actually, directly collectively self-defeating, but I'll skip Parfit's distinction between direct and indirect.)

Maybe being collectively self-defeating is fine—Parfit does not say it's grounds for rejecting . What's special about this argument is that it is the first argument in the book—not the most important one—where an LDT-inspired move goes somewhere interesting.

I'll begin by paraphrasing the argument, and then we'll reconsider from a perspective informed by LDT.

Parfit's argument: S in the prisoner's dilemma

The argument that is collectively self-defeating begins by considering two theorists in a prisoner's dilemma, with which I assume you are familiar.

recommends defection, since you cannot affect what your opponent does, and regardless of what they do, it is better to defect.

Therefore both players defect, and each player is worse off than they would have been if they had both cooperated.

is collectively self-defeating, since by following 's recommendation to defect, both players have done worse by their -given aim than if they had both cooperated.

Is it so clear that they both defect? Let's concede that they do. I'm going to apply the basic principle of LDT not by disputing this argument about , but by proposing a new theory, a modification of .

The self-referential self-interest theory S²

The new theory, , gives you the same aim as , but a different policy. It recommends taking the action that it would be best for you for to recommend.

I call this the self-referential self-interest theory. We could abbreviate that to . Thus the cutesy name , even though the 's don't stand for the same thing.

Let's consider a prisoner's dilemma between theorists, and assume that they have common knowledge that they are both theorists. What does recommend?

S² recommends cooperation if we assume the right counterpossibles

Assume that for each action, if recommends that action, then both players will take it. Or, if it had recommended it, then both players would have taken it. These are the counterpossible implications we need for this problem.

If we have these counterpossibles, it's straightforward to derive 's recommendation. If recommends that you defect, then both players defect. Whereas if recommends that you cooperate, then both players cooperate. This is better for you than if both players defect, so recommends that you cooperate.

Therefore the outcome is mutual cooperation, and Parfit's argument that is collectively self-defeating doesn't go through for .

Where can we get S²'s counterpossibles?

But why assume that whatever recommends, both players will do (and that if it had recommended otherwise, then both players would have done that instead)? Because the game is symmetrical: it looks the same from the perspective of each player. So both players should get the same recommendation from . And since they're both theorists, they both follow that recommendation.

We want that reasoning to go through, but it seems hard for any formal theory to work like that. I started this post by telling you that counterpossibles present theoretical difficulties, and next I'll explain how that manifests in this example.

The problematic counterpossible for S² in the prisoner's dilemma

The problematic counterpossible implication in this scenario is the statement: "if had recommended defection, then both players would have defected."

We can't do without this statement. It's why recommends cooperation.

But it's problematic because if cooperation logically follows from our assumptions about the prisoner's dilemma and , then we cannot consistently reason from the premise that recommends defection.

To see the inconsistency clearly: first, assume that you have some valid argument that recommends cooperation. Now, imagine another argument which begins with the premise that recommends defection. Insert after that premise all the steps of argument , ending with the statement that recommends cooperation. With that in conjunction with the premise, concludes that recommends defection and that it recommends cooperation. , then, is a valid argument from its premise to a contradiction.

If we cannot consistently reason from the premise that recommends defection, how can we assert what would happen in that case? Or, considering the principle of explosion, how can we avoid asserting everything: that if recommends defection then both players defect, and they cooperate, and get acceptance letters from Hogwarts?

That's why counterpossibles present more serious issues than ordinary counterfactuals. There is not just a fact but a logical argument contradicting the premise.

Conclusion

One might wonder, can you just ignore Newcomb's problem and the whole body of decision theory literature that it spawned? I think the answer is no. The kind of situations that logical decision theory is supposed to address come up naturally in the course of philosophical arguments.

When reading Reasons and Persons, the argument above was the first place in the book where, being familiar with LDT, I saw an important unexplored path that seemed to undermine the conclusion. There are more cases like that in the book, although I haven't read far enough into it to judge what that ultimately means for Parfit's main theses.

But I hope I've also communicated to you why LDT is at best a work in progress, and why some would consider it just nonsense. Theories in the LDT family include timeless decision theory and functional decision theory, and as I understand it, both of these put in the counterpossibles by hand the same way I did in this post. I won't really be satisfied with arguments like the one in this post until I can follow the logic step by step in a consistent formal theory, all the way from the assumptions about the game to the conclusion about the theory's recommendation.

But Newcomb’s problem sounds like a science fiction story, and I’m not convinced by the case that analogous problems are part of our ordinary social lives.

I mean, consider Rock Paper Scissors. Your opponent moves first, their move is done but unknown to you. You can pass on making a move or not. Do you, like a normal human, take into account the skill of your opponent before deciding whether to play? If Yes, then this eliminates CDT.

I mean, consider Rock Paper Scissors. Your opponent moves first, their move is done but unknown to you. You can pass on making a move or not. Do you, like a normal human, take into account the skill of your opponent before deciding whether to play? If Yes, then this eliminates CDT.

If I can pick a mixed strategy, it is impossible for my opponent to be more skilled at rock-paper-scissors than me.

I mean, CDT specifically recommends not picking mixed strategy! Isn't it crazy?

CDT thinks just straight up picking at least one option has positive utility, if you use RNG you would get less utility according to CDT than picking up to your best judgment of your oponent.

CDT is incapable of thinking its final choice can be predicted, and so does not take options to make it less predictable, as it thinks it's alredy impossible to predict.

CDT doesn't recommend not picking a mixed strategy here. If the payouts are the same, a mixed strategy is optimal.

If you can pick a mixed strategy, your final choice cannot be predicted.

No. I think you use "CDT" as "normal decision theory without weird acausual stuff", but it is a very specific theory, that specifically says not to pick mixed strategy, as you lose money with it.

EDT does recommend to use a mixed strategy, to be fair. EDT is the one which deals with game theory better, but still not very well, see XOR blackmail problem.

This is wrong. Just factually. I suggest you read e.g. Joyce on causal decision theory for a good understanding of its formal meaning.

Causal decision theory recommends mixed strategies in unstable equilibrium.

See, e.g. page 474 here: https://eprints.ukh.ac.id/id/eprint/240/1/2016_Book_ReadingsInFormalEpistemology.pdf#page=474

I mean it's not plain CDT, it's variant of CDT with ratification? I'm not sure how it should work, it's weird. You are like, searching over belief states that will lead to desirable conclusion?

E.g. your skillful opponent already made the move, but it's unknown to you, and there are 4 buttons: Rock, Paper, Scissors, Random_Choice.

It's obvious that at least one of Rock, Paper, Scissors has more CDT expected utility than Random_Choice, given that the opponent's move is already fixed and you have to have some belief about its actual state, and no matter how small the difference in your belief in its state, it breaks the tie.

I mean it's not plain CDT, it's variant of CDT with ratification?

Ratification just means a solution to a decision theory problem. A problem that is ratifiable is a problem that has a stable solution. Scroll up and he discussing ratification as a concept more.

Joyce is not talking about some other form of CDT.

It's obvious that at least one of Rock, Paper, Scissors has more CDT expected utility than Random_Choice

That is the case if and only if you believe one option is more likely to have been played then the others. If you think there is a 70% chance your opponent played scissors, then in that situation you should always play rock. In that situation, playing rock is ratifiably the best choice.

In the case where you are playing another rational agent in a normal-form game, the only ratifiable solution is for both players to have equal chances of playing rock paper or scissors, i.e. they adopt a mixed strategy.

If you think there is a 70% chance your opponent played scissors, then in that situation you should always play rock.

Any, however small divergence from equal credences on your opponent's fixed move, leads you to playing the counter, instead of pressing random move button.

But good opponents are making moves that are in fact dependent on your move, and as you drop that EDT style update, where after making move your estimate of opponents move would change to the counter to yours, and utility along it, you get screwed.

CDT thinks it's very small positive utility, but EDT correctly thinks it's large negative utility.

Any, however small divergence from equal credences on your opponents fixed move, leads you to playing the counter

Not however small, it depends on the payouts as to the mixed strategy you should adopt, just like what any game theorist would say. It you have a greater than not odds, yes you should play the move that you expect to win. If there is a dominant pure strategy, CDT recommends that.

But good opponents are making moves that are in fact depended on your move,

If your opponent is also a rational agent, the only stable solution is for both players to randomly select rock-paper-scissors with equal frequency.

But good opponents are making moves that are in fact depended on your move

They are making moves dependent on what they expect you to do. Unless they are time travelers. You know they make moves depending on what you do, and they know you know that and so on. If it is a normal-form game, that means that any solution other than randomly picking is not a stable solution. Your only stable solution is to pick randomly.

I mean, where do you see perfect agents?

If you see patterns in their behavior, you can predict them, and they would be foolish to play with you, unless by pressing random_choice. If they see patterns in your behavior they can predict you, and you would be foolish to play with them, unless by pressing random_choice.

If you use CDT, and press random_choice against imperfect agent, you leave money on the table, according to CDT.

I mean, where do you see perfect agents?

Rational agents is a required assumption for any decision theory.

If you see patterns in their behavior, you can predict them, and they would be foolish to play with you, unless by pressing random_choice

If you are a rational agent and your opponent is not, it is possoble for a stable solution where you win more often than not. The only way to get you to pick something that would have you losing more often than not (w/o playing with payoffs) is if your expectations about the outcomes are wrong. The agent with wrong predictions losing to the one with better predictions is unremarkable.

Rational agents is a required assumption for any decision theory

No? CDT definition makes no references to agents, nor rational agents. It sees the world, and itself, and makes no distinction of dumb matter and agents.

No? CDT definition makes no references to agents, nor rational agents. It sees the world, and itself, and makes no distinction of dumb matter and agents.

What do you think decision theories are? The entire point of a decision theory is to describe what decisions rational agents aught to make. If there is no agent to make a decision, there is no decision theory. You can see page 5 of the book i previously linked for a decent definition of decision theory. Or the section on decision theory beginning on page 351.