Newcomb's problem happened to me

by Academian3 min read26th Mar 201099 comments

43

Decision TheoryNewcomb's ProblemRelationships (Interpersonal)
Frontpage

Okay, maybe not me, but someone I know, and that's what the title would be if he wrote it.  Newcomb's problem and Kavka's toxin puzzle are more than just curiosities relevant to artificial intelligence theory.  Like a lot of thought experiments, they approximately happen.  They illustrate robust issues with causal decision theory that can deeply affect our everyday lives.

Yet somehow it isn't mainstream knowledge that these are more than merely abstract linguistic issues, as evidenced by this comment thread (please no Karma sniping of the comments, they are a valuable record).  Scenarios involving brain scanning, decision simulation, etc., can establish their validy and future relevance, but not that they are already commonplace.  For the record, I want to provide an already-happened, real-life account that captures the Newcomb essence and explicitly describes how.

So let's say my friend is named Joe.  In his account, Joe is very much in love with this girl named Omega… er… Kate, and he wants to get married.  Kate is somewhat traditional, and won't marry him unless he proposes, not only in the sense of explicitly asking her, but also expressing certainty that he will never try to leave her if they do marry

Now, I don't want to make up the ending here.  I want to convey the actual account, in which Joe's beliefs are roughly schematized as follows: 

  1. if he proposes sincerely, she is effectively sure to believe it.
  2. if he proposes insincerely, she will 50% likely believe it.
  3. if she believes his proposal, she will 80% likely say yes.
  4. if she doesn't believe his proposal, she will surely say no, but will not be significantly upset in comparison to the significance of marriage.
  5. if they marry, Joe will 90% likely be happy, and will 10% likely be unhappy.

He roughly values the happy and unhappy outcomes oppositely:

  1. being happily married to Kate:  125 megautilons
  2. being unhapily married to Kate:  -125 megautilons.

So what should he do?  What should this real person have actually done?1  Well, as in Newcomb, these beliefs and utilities present an interesting and quantifiable problem…

  • ExpectedValue(marriage) = 90%·125 - 10%·125 = 100,
  • ExpectedValue(sincere proposal) = 80%·100 = 80,
  • ExpectedValue(insincere proposal) = 50%·80%·100 = 40.

No surprise here, sincere proposal comes out on top.  That's the important thing, not the particular numbers.  In fact, in real life Joe's utility function assigned negative moral value to insincerity, broadening the gap.  But no matter; this did not make him sincere.  The problem is that Joe was a classical causal decision theorist, and he believed that if circumstances changed to render him unhappily married, he would necessarily try to leave her.  Because of this possibility, he could not propose sincerely in the sense she desired.  He could even appease himself by speculating causes2 for how Kate can detect his uncertainty and constrain his options, but that still wouldn't make him sincere

Seeing expected value computations with adjustable probabilities for the problem can really help feel its robustness.  It's not about to disappear.  Certainties can be replaced with 95%'s and it all still works the same.  It's a whole parametrized family of problems, not just one. 

Joe's scenario feels strikingly similar to Newcomb's problem, and in fact it is:  if we change some probabilities to 0 and 1, it's essentially isomorphic: 

  1. If he proposes sincerely, she will say yes.
  2. If he proposes insincerely, she will say no and break up with him forever.
  3. If they marry, he is 90% likely to be happy, and 10% likely to be unhappy.

The analogue of the two boxes are marriage (opaque) and the option of leaving (transparent).  Given marriage, the option of leaving has a small marginal utility of 10%·125 = 12.5 utilons.  So "clearly" he should "just take both"?  The problem is that he can't just take both.  The proposed payout matrix would be:

Joe \ Kate
Say yes
Say no
Propose sincerely
Marriage Nothing significant
Propose insincerely
Marriage + option to leave Nothing significant

The "principal of (weak3) dominance" would say the second row is the better "option", and that therefore "clearly" Joe should propose insincerely.  But in Newcomb some of the outcomes are declared logically impossible.  If he tries to take both boxes, there will be nothing in the marriage box.  The analogue in real life is simply that the four outcomes need not be equally likely

So there you have it.  Newcomb happens.  Newcomb happened.  You might be wondering, what did the real Joe do

In real life, Joe actually recognized the similarity to Newcomb's problem, realizing for the first time that he must become updateless decision agent, and noting his 90% certainty, he self-modified by adopting a moral pre-commitment to never leaving Kate should they marry, proposed to her sincerely, and the rest is history.  No joke!  That's if Joe's account is accurate, mind you.

 


Footnotes:

1 This is not a social commentary, but an illustration that probabilistic Newcomblike scenarios can and do exist.  Although this also does not hinge on whether you believe Joe's account, I have provided it as-is nonetheless. 

2 If you care about causal reasoning, the other half of what's supposed to make Newcomb confusing, then Joe's problem is more like Kavka's (so this post accidentally shows how Kavka and Newcomb are similar).  But the distinction is instrumentally irrelevant:  the point is that he can benefit from decision mechanisms that are evidential and time-invariant, and you don't need "unreasonable certainties" or "paradoxes of causality" for this to come up. 

3 Newcomb involves "strong" dominance, with the second row always strictly better, but that's not essential to this post.  In any case, I could exhibit strong dominance by removing "if they do get married" from Kate's proposal requirement, but I decided against it, favoring instead the actual account of events.

43

99 comments, sorted by Highlighting new comments since Today at 8:38 PM
New Comment
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

I predict, with probability ~95%, that if Joe becomes unhappy in the marriage, he and Kate will get divorced, even though Joe and Kate, who is not as powerful a predictor as Omega, currently believe otherwise. Joe is, after all, running this "timeless decision theory" on hostile hardware.

(But I hope that they remain happy, and this prediction remains hypothetical.)

Your prediction is overconfident. Less than 95% of unhappy marriages end in divorce.

1JGWeissman11yPerhaps, I didn't look up any statistics. The "~" in "~95%" was supposed to indicate my meta uncertainty that this is the precise strength of belief an ideal rationalist should have given evidence observed by me. I am confident that I am closer to the ideal probability than 0% as believed by Kate. Apparently 45% to 50% of first marriages in America end in divorce [http://www.divorcestatistics.org/], but this does not account for whether the marriages were unhappy. Do you have a source for your assertion? I have not found anything with a quick Google search.
1Unknowns11yhttp://www.ultimate-self.com/unhappy-marriage-statistics/ [http://www.ultimate-self.com/unhappy-marriage-statistics/]
-2mutterc10y100% of marriages end in divorce or death.
9wedrifid10y100% of marriages that have ended ended in divorce or death.
3mutterc10yGood point; if we conquer death then there may be some marriages that do not end. It'd be interesting to see if people move towards near-universal divorce, sci-fi-novel-style limited-term marriages, or find ways to develop infinite-term compatibility. Or stop pairing up (inconceivable to present-day humans, but such is the nature of a Singularity).
0wedrifid10yThat definitely would be interesting. It would perhaps be an indicator of preferences, as opposed to the current indicator of capability. If you have tools that can alter your mind you can cheat.
2XFrequentist10yHistorically.
1AdeleneDawner10yWhich of those does it count as when one of the parties just leaves and becomes unfindable?
2mutterc10yMy understanding is that you're still married until one of you goes and gets a divorce, but I can't admit to having researched such a thing.
1AdeleneDawner10yI suspect that getting a divorce requires some minimal amount of input from both parties - if I remember correctly I had to sign something saying that I'd received some paperwork, when mine happened, in order for it to go through. I suspect that in the case I posited, the non-disappearing person would be able to get the disappearing person declared dead after a certain period of time, which doesn't strictly require that the disappearing person be dead, and then remarry. If that's accurate, that'd be a third option.
2wedrifid10y100% of marriages that have ended have ended in divorce or legal death? Where does 'annulment' fit into things? Is that when it is decided to just pretend the marriage never existed in the first place.
1thomblake10yYes. In the Catholic Church, a "declaration of nullity" was nearly a loophole to not being able to get divorced. Basically, there were certain preconditions that were assumed to hold when getting married, and if it turns out any of those preconditions did not actually obtain, then the marriage never actually happened. For example, it is assumed that the couple wants to have children; if it turns out that one party never intended to have children, that can be grounds for a declaration of nullity. Several legal jurisdictions have adopted this idea, but it makes little sense when one can just get divorced and there are not strict preconditions for marriage. Wikipedia: Annulment [http://en.wikipedia.org/wiki/Annulment]
0Sniffnoy10yAs long as we're picking nits, in some places marriages can also be annulled (though of course they will insist that this is retroactive, and for some purposes it is).
0mutterc10yThat's what I understand; an annulment means the marriage never happened. (E.g. if it's been "consummated" then annulment is not an option. I wonder how that interacts with modern pre-consummated marriages?)
7bentarm11yI predict with probability ~95% that if statisticians had arbitrarily decided many years ago to use 97% instead of 95% as their standard of proof, then all appearances of 95 and 97 in this comment would be reversed.
4JGWeissman11yHow does it change your prediction to learn that I was not considering statisticians' arbitrary standard of proof, but I was thinking about numbers in base ten, and I had considered saying ~90% instead?
1shokwave10yNot much for me. I think it about six times more likely that you used base ten numbers to "get to" 95% than it is you came to 95% by coincidence.
3Academian11yYeah, it's a big open problem if some humans can precommit or not, making the issue of its value all the more relevant.
7JGWeissman11yNo, it's not. I don't see any reason to believe that humans can reliably precommit, without setting up outside constraints, especially over time spans of decades. What you have described is not Newcomb's problem. Take what taw said [http://lesswrong.com/lw/1zw/newcombs_problem_happened_to_me/#comments], and realize that actual humans are in fact in this category:
2Academian11yadded: Try playing with the parameters. Maybe Kate only wants 90% certainty from Joe, and Joe is only 80% sure he'll be happy. Then he doesn't need a 100% precomitment, but only some kind of partial deterrent, and if Kate requires that he not resort to external self-restrictions, he can certainly self-modify partial pre-commitments into himself in the form of emotions. Self-modification is robust, pre-commitment is robust, its detection is robust... these phenomena really aren't going anywhere.
2JGWeissman11yReplacing the certainties with 95% still does not reflect reality. I don't think Kate can assign probability to whether she and Joe will get divorced any better than by taking the percentage of marriages, possibly in some narrow reference class they are part of, that end in divorce [http://wiki.lesswrong.com/wiki/Outside_view]. Even if Joe can signal that he belongs to some favorable reference class, it still won't work [http://lesswrong.com/lw/1ws/the_importance_of_goodharts_law/].
4tut11yIf they are rational enough to talk about divorce in order to avoid it, then he can make an economic commitment by writing a prenup that guarantees that any divorce becomes unfavorable. Of course, only making it relatively unfavorable will give her an incentive to leave him, so it is better if a big portion of their property is given away or burned in case of a divorce.
0JGWeissman11yYes, that is a strategy they can take, However, that sort of strategy is unnecessary in Newcomb's problem, where you can just one-box and find the money there without having made any sort of precommitment.
2tut11yI think that the translation to Newcombe's was that committing == one boxing and hedging == two boxing.
1JGWeissman11yThis mapping does not work. Causal Decision Theory would commit (if available) in the marriage proposal problem, but two box in Newcomb's problem. So the mapping does not preserve the relationship between the mapped elements. This should be a sanity check for any scenario proposed to be equivalent to Newcomb's problem. EDT/TDT/UDT should all do the equivalent of one-boxing, and CDT should do the equivalent of two-boxing.
2Nick_Tarleton11yCDT on Newcomb's problem would, if possible, precommit to one-boxing as long as Omega's prediction is based on observing the CDT agent after its commitment. CDT in the marriage case would choose to leave once unhappy, absent specific precommitment. So that exact mapping doesn't work, but the problem does seem Newcomblike to me (like the transparent-boxes version, actually; which, I now realize, is like Kavka's toxin puzzle without the vagueness of "intent".) (ETA: assuming that Kate can reliably predict Joe, which I now see was the point under dispute to begin with.)
0JGWeissman11yWould you care to share your reasoning? What is your mapping of strategies, and does it pass my sanity check? (EVT two-boxes on the transparent boxes variation.)
0Nick_Tarleton11yone-box <=> stay in marriage when unhappy two-box <=> leave marriage when unhappy precommit to one-boxing <=> precommit to staying in marriage In both this problem and transparent-boxes Newcomb: * you don't take the action under discussion (take boxes, leave or not) until you know whether you've won * if you would counterfactually take one of the choices if you were to win, you'll lose * TDT and UDT win * CDT either precommits and wins or doesn't and loses, as described in my previous comment (I'm assuming that Kate can reliably predict Joe. I didn't initially realize your objection might have more to do with that than the structure of the problem.)
1Jack11yIf Jack and Kate were already married it really would make no sense for Jack to not get a divorce just because Kate would have never married him had she suspected he would. CDT wins, here. The fact that CDT tells Jack to precommit now doesn't make it Newcomblike. Precommiting is a rational strategy in lots of games that aren't Newcomb like. The whole point of Newcomb is that even if you haven't precommitted, CDT tells you the wrong thing to do once Omega shows up.
0Nick_Tarleton11yAs I said, I assumed that Kate = Omega.
0Jack11yEven if that assumption is fair (since it obviously isn't true I'm not sure why we would make it**) we're still entering the scenario too early. It's like being told Omega is going to offer you the boxes a year before he does. Jack now has the opportunity to precommit, but Omega doesn't give you that chance. ** I'm sure glad my girlfriend isn't a superintelligence that can predict my actions with perfect accuracy! Am I right guys?!
0Nick_Tarleton11yPoint taken; the similarity is somewhat distant. (I made that assumption to show the problem's broadly Newcomblike structure, since I wrongly read JGWeissman as saying that the problem never had Newcomblike structure. But as you say, there is another, more qualitative difference.)
0JGWeissman11yYes, that is where my objection lies. ETA: And the fact that in Newcomb's problem, there is no opportunity after learning about the problem to precommit, the predictions of your behavior have already been made. So allowing precommitment in the marriage proposal problem sidesteps the problem that would be Newcomb like if Kate were a highly accurate predictor.
0[anonymous]11yCausal decision theory precommits to one-boxing on Newcomb if it can and if Omega's prediction is based on observation of the CDT agent after its opportunity to precommit.
-1JGWeissman11yWhy is the parent comment being voted down, and its parent being voted up, when it correctly refutes the parent? Why is the article itself being voted up, when it has been refuted? Are people so impressed by the idea of a real life Newcomb like problem that they don't notice, even when it is pointed out, that the described story is not in fact a Newcomb like problem?
5wedrifid11yI voted it up because it is a good article. The claim "this situation is a problem of the class Newcomblike" has been refuted. If Academian had belligerently defended the 'It's Newcomblike' claim in response to correction I would have reversed my upvote. As it stands the discussion both in the original post and the comments are useful. I expect it has helped clarify how the situation as it is formalized here differs from Newcomb's problem and what changes the scenario would need to actually be a Newcomblike problem. In fact, that is a follow up post that I would like to see. Ease up. The "it's not actually Newcomblike" comments are being upvoted. People get it. It's just that sometimes correction is sufficient and a spiral of downvotes isn't desirable.
-2JGWeissman11yIt is an article in which poor thought leads to a wrong conclusion. I don't consider that "good". I wouldn't say he was belligerent, but earlier in this thread he seemed to be Fighting a Rearguard Action Against the Truth [http://lesswrong.com/lw/u8/fighting_a_rearguard_action_against_the_truth/], first saying, "it's a big open problem if some humans can precommit or not", and then saying the scenario still works if you replace certainties with high confidence levels, with those confidence levels also being unrealistic. I found "Self-modification is robust, pre-commitment is robust, its detection is robust... these phenomena really aren't going anywhere." to be particularly arrogant. He seems to have dropped out after I refuted those points. My standard for changing this article from bad to sort of ok, would require an actual retraction of the wrong conclusion. As it stands, someone can be led astray by reading just the article and not the comments. Not as much as the article. And this comment [http://lesswrong.com/lw/1zw/newcombs_problem_happened_to_me/1trl], which refuted a wrong argument that the scenario really is Newcomb's problem, at the time I asked that question, was at -2. I am not saying everyone should vote it down so Academian loses so much karma he can never post another article. I think a small negative score is enough to make the point. A small positive score would be appropiate if he made a proper retraction. +27 is too high. I don't think articles should get over +5 without the main point actually being correct, and they should be incredibly thought provoking to get that high. I am also wary of making unsupportable claims that Newcomb's problem happens in real life, which can overshadow other reasons we consider such problems, so these other reasons are forgotten when the unsupportable claim is knocked down.
0wedrifid11yI can empathise with your point of view here. Perhaps the fact that people (including me) still appreciate the post despite it getting the game theory discussion wrong is an indication that we would love to see more posts on 'real life' applications of decision theory!
4pjeby11yThat depends entirely on what characteristics you consider to be most "Newcomb like". From an emotional point of view, the situation is very "Newcomb like", even if the mathematics is different.
-1JGWeissman11yThis sounds like a fully general excuse to support any position. What is this emotional view? If the emotions disagree with the logical analisys, why aren't the emotions wrong? Correct emotions should be reactions to the actual state of reality.
3pjeby11yWe seem to be having a language difficulty. By "emotional point of view", I mean that there are similarities in the human emotional experience of deciding Newcomb's problem and the marriage proposal problem.
2wedrifid11y(Agree) Evolution built in (a vague approximation of) one boxing into our emotional systems. Humans actually can commit, without changing external payoffs. It isn't a bullet proof commitment. Evolution will also try to create effective compartmentalization mechanisms so that humans can maximise signalling benefit vs actual cost to change later.
0JGWeissman11yOn a timescale of decades, the commitment has hardly any strength at all.
1wedrifid11yFortunately, it isn't meant to be. In a crude sense the emotions are playing a signalling game on a timescale of months to a couple of years. That the emotions tell us they are talking about 'forever' is just part of their game.
-1JGWeissman11yThen you agree that Joe's commitment is not a good indicator that he will stay in the marriage for decades, so Joe did not get what he wanted by allowing Kate to make an accurate prediction that he will do what she wants? Why, when we are discussing a problem that requires commitment on the scale of many decades, did you bring up that humans can make commitments up to maybe a couple of years?
0wedrifid11yI haven't said any such thing. Joe and Kate are counterfactual in as much as organic emotional responses were simplified to Kate having a predictive superpower and Joe the ability to magically (and reliably) self modify. Two real people would be somewhat more complex and their words and beliefs less literally correlated with reality. The basic mechanism of conversation requires that I follow the flow rather than making every comment as a reply to the original post. When I learned that from a book there was an analogy about tennis involved which I found helpful.
-1JGWeissman11yYou did say "In a crude sense the emotions are playing a signaling game on a timescale of months to a couple of years." And the scenario does involve predictions of events which take place over the time scale of decades. Do you disagree with my assessment of the time scales, or do you somehow disagree with the conclusion? The scenario was presented as something that actually happened, to two real people, with Kate's beliefs literally correlating with reality.
0wedrifid11yI am comfortable with leaving my previous statements as they stand. I think we are taking a somewhat different approach to discussion here and remind myself that my way of thinking may not be strictly better than yours, merely different (P vs J) [http://lesswrong.com/lw/1xq/let_there_be_light/1s0x].
-3JGWeissman11yIf that is true (which I am not sure of, and it is hard to tell since your vague claim doesn't specify the common emotional experience), isn't that just an example of how our human emotions are misleading?
-1JGWeissman11yI hate to have to ask this sort of question again, but would the downvoters care to explain why it is wrong for me to ask why PJEby thinks we should draw conclusions about decision theory from human emotional reactions, or to ask what particular emotional reactions he is talking about?
3reaver12111yThat's the reason why I never get why people are against marriage contracts. Even ignoring the inherent uncertainty of love & marriage, if I walk under a bus tomorrow and lose for example all empathy due to brain damage, my current self would wish you to divorce future psychopath-me as quickly as possible. As for the OP, good article. If anyone ever asks why I spend my time theorizing away over 'impossible' things like AI or decision theory I can use this as an example.
3PhilGoetz11yDid you mean to say you don't understand why people are in favor of marriage contracts? I don't see how the marriage contract helps in the bus example.
3reaver12111ySorry, I used the wrong terminology. I meant an prenuptial agreement. The bus example was to show that even if you precommit there is always the possibility that you will change your mind (i.e. in this case by losing empathy). I used the extreme method of brain damage because it's completely out of your control. You cannot precommit on not being run over by a bus.
0[anonymous]11yI'm pretty sure thinking about scenarios with low probability, especially over the long term, is considered "unromantic". Disclaimer: commenter is generally anti-marriage and not typically romantic.

In real life, Joe actually recognized the similarity to Newcomb's problem, realizing for the first time that he must become timeless decision agent, and noting his 90% certainty, he self-modified by adopting a moral pre-commitment to never leaving Kate should they marry, proposed to her sincerely, and the rest is history.

It would be a (probabilistic approximation of a) Newcomb problem when considered without the ability to precommit or otherwise sabotage the future payoff for one of your future options. Having that option available makes the problem o... (read more)

1Academian11yYes, it was more Newcomblike before Joe realized his ability to pre-commit (or "hypothetically self sabotage" as you might call it), and less Newcomblike afterwards.

Uh, someone having a script for your life that they require you to fit does not make them Omega - it just means they are attempting to dominate you and you are going along with it, like in many ordinary relationships. Admittedly I may be biased myself from having been burnt, but this is a plain old relationship problem, not Newcomb's problem. The answer is not a new decision theory, but to get out of the unhealthy and manipulative relationship.

I concur with JGWeissman's prediction. I just don't find it credible that time-binding apes, faced with the second... (read more)

0duckduckMOO9yThere are (human) apes that will. I think that you're underestimating how -for lack of a better word-, deontologically/morally, some people see things. Also (or maybe this is what I mean by deontologically) how much of a self image someone can tied up in being a person who doesn't break promises or how -for lack of a better word (that I can think of) literally people can think of breaking specific commitments. Well, except the grinning. In any case if the marriage goes badly Kate is free to leave so unless it goes so badly she wants to string him along to torment him he can always ask her to end it. also this, "I just don't find it credible that time-binding apes, faced with the seconds of their life ticking away in misery, will just grin and put up with it without massive outside pressure." They do.

This seems better described as a variant of the traditional paradox of hedonism. That is, some goals (e.g. long term happiness) are best achieved by agents who do not explicitly aim only at this goal, and who can instead be trusted to keep to their commitments even if it turns out that they'd benefit from defecting.

2ata11yThat doesn't really sound like a paradox, just more evidence that people are very suboptimal optimizers. If the goal is long-term happiness, and some actions are more conducive to that than the actions most people come up with when aiming for long-term happiness, then that only indicates we're bad at reasoning about long-term goals.
0RichardChappell11yHmm, I think you've missed something if you can't tell this apart from the general phenomenon of being "very suboptimal optimizers". The problem isn't that bad consequences result from our seeking pleasure ineptly. It's instead that bad consequences result from our seeking pleasure (even if all our means-end calculations are perfectly accurate). I agree that it's a rather loose use of the term 'paradox', but this is the standard term for the phenomenon, dating back more than a century now. For more background, see the Stanford encyclopedia [http://plato.stanford.edu/entries/hedonism/#1] and wikipedia [http://en.wikipedia.org/wiki/Paradox_of_hedonism]. (Parfit's 'rational irrationality [http://www.philosophyetc.net/2005/03/rational-irrationality.html] ' is also related.)
1ata11yThat sounds like a contradiction. If you're perfect at doing means-end calculations, and the best way to attain pleasure or happiness is something other than seeking it directly, then your calculations will tell you that, and you will do it. Maybe I'm missing something, but this sounds more like an aesop about the perils of hedonism, and I'm not sure it would apply to perfect decision-makers.
3RichardChappell11yIt's no contradiction. Perfect means-end calculations merely ensures that you'll choose the best of the options available given that you've made a means-end calculation. But you might have different (and better) options if you never made any such calculation. (For a crude illustration, imagine that God exists and will reward people who never make any attempt at instrumental reasoning.) By the time your calculations tell you that you never should have calculated in the first place, it's too late.
0HopeFox10yPerfect decision-makers, with perfect information, should always be able to take the optimal outcome in any situation. Likewise, perfect decision-makers with limited information should always be able to choose the outcome with the best expected payoff under strict Bayesian reasoning. However, when the actor's decision-making process becomes part of the situation under consideration, as happens when Katemega scrutinises Joe's potential for leaving her in the future, then the perfect decision-maker is only able to choose the optimal outcome if he is also capable of perfect self-modification. Without that ability, he's vulnerable to his own choices and preferences changing in the future, which he can't control right now. I'd also like to draw a distinction between a practical pre-commitment (of the form "leaving this marriage will cause me -X utilons due to financial penalty or cognitive dissonance for breaking my vows"), and an actual self-modification to a mind state where "I promised I would never leave Kate, but I'm going to do it anyway now" is not actually an option. I don't think humans are capable of the latter. An AI might be, I don't know. Also, what about decisions Joe made in the past (for example, deciding when he was eighteen that there was no way he was ever going to get married, because being single was too much fun)? If you want your present state to influence your future state strongly, you have to accept the influence of your past state on your present state just as strongly, and you can't just say "Oh, but I'm older and wiser now" in one instance but not the other. Without the ability to self-modify into a truly sincere state wherein he'll never leave Kate no matter what, Joe can't be completely sincere, and (by the assumptions of the problem) Kate will sense this and his chances of his proposal being accepted will diminish. And there's nothing he can do about that.
1Bongo10yI have to note that an agent using one of the new decision theories sometimes discussed around here, like UDT, wouldn't leave Katemega and wouldn't need self-modification or precommitment to not leave her.

It's not a Newcomb problem. It's a problem of how much his promises mean.

Either he created a large enough cost to leaving if he is unhappy, in that he would have to break his promise, to justify his belief that he won't leave; or, he did not. If he did, he doesn't have the option to "take both" and get the utility from both because that would incur the cost. (Breaking his promise would have negative utility to him in and of itself.) It sounds like that's what ended up happening. If he did not, he doesn't have the option to propose sincerely, since he knows it's not true that he will surely not leave.

1Academian11yCreating internal deterrents is a kind of self modification, and you're right that it's a way of systematically removing or altering one's options.

If you're calling the potential bride in your scenario Kate, you should really have called her suitor Petruchio :-)

It's an interesting situation, and I can see the parallel to Newcombe's Problem. I'm not certain that it's possible for a person to self-modify to the extent that he will never leave his wife, ever, regardless of the very real (if small) doubts he has about the relationship right now. I don't think I could ever simultaneously sustain the thoughts "There's about a 10% chance that my marriage to my wife will make me very unhappy" and "I will never leave her no matter what". I could make the commitment financially - that, even if the marri... (read more)

How about a pre-nup and polyamory boxing?

1TimFreeman10yWhat is that? Does it require padded gloves?
2wedrifid10yNo, it's when you keep spares in storage till you need them.

If I could attempt to summarise my interpretation of the above:

Joe realises that the best payout comes from proposing sincerely even though he is defined to be insincere (10% probability of surely breaking his promise to never try and leave her if they marry). He seeks a method by which to produce an insincere sincere proposal.

As sincerity appears to be a controllable state of mind he puts himself in the right state, making him appear temporarily sincere and thus aiming for the bigger payout.

As you have not assigned any moral or mental cost associated with... (read more)

One dissimilarity from Newcomb's is that the marginal utility of spouses decreases faster than the marginal utility of money, and moreover many potential spouses are known to exist. (I.e., Joe can just walk away and find someone more reasonable to marry for relatively small utility cost.)

If precommitment is observable and unchangeable, then order of action is:

  • Joe: precommit or not
  • Kate: accept or not - knowing if Joe precommitted or not
  • Joe: breakup (assuming no precommitment)

If precommitment is not observable and/or changeable, then it can be rearranged, and we have:

  • Kate: accept or not - not having any clue what Joe did
  • Joe: breakup or not

Or in the most complex situation, with 3 probabilistic nodes:

  • Joe: precommit or not
  • Nature: Kate figures out what Joe did correctly or not
  • Kate: accept or not
  • Nature: Marriage happy or unhappy
  • N
... (read more)
7wedrifid11yIt is the Newcomb Problem. It may be tricky and counter-intuitive but it isn't a paradox. More importantly The Newcomb Problem does not rely on a causal loop. Some form of reliable prediction is necessary but that does not imply a causal loop.
-4taw11yreliable prediction = causal loop
8Nick_Tarleton11yThis isn't the case when you predict that a calculator on which you input 2+2 will output 4. Why would it necessarily be the case for predicting a person?
0PhilGoetz10yWhy was this voted down so hard? Please explain. It sounds reasonable to me.
4PhilGoetz11yIf Joe believes that his precommitment is inviolable, or even that it affects the probability of him breaking up later, then it appears to him that he is confronted with a causal loop. His decision-making program, at that moment, addresses Newcomb's problem, even if it's wrong in believing in the causal loop. But I think this only proves that flawed reasoners may face Newcomb's problem. (It might even turn out that finding yourself facing Newcomb's problem proves your reasoning is flawed.) It's still interesting enough to up-vote.
1Academian11yMy pre-sponse to this is in footnote 2:
0taw11yThere is no need for time-invariance. The most generic model (2 Joe nodes; 1 Kate note; 3 Nature nodes) of vanilla decision theory perfectly explains the situation you're talking about - unless you postulate some causal loops.
0Academian11yIs that not the simplicity you're interested in?
-1taw11yAnd in Kavka's problem there's no paradox unless we assume causal loops (billionaire knows now if you're going to decide to drink the toxin or not tomorrow), or leave the problem ambiguous (so can you change or mind or not?).
3Academian11yYou'll notice I didn't once use the word "paradox" ;)