Was this motivated by Nick Bostrom's paper? If not, you might enjoy it--it also discusses the idea of splitting a consciousness into copies by duplicating identical computational processes.
Yup. In short, I think there is a very simple place when a slowly-dividing computer should start acting as two agents, and that is when, in the sleeping beauty problem, I make two bets with them rather than one.
Hmm. If we use that sort of functional definition, does that mean that if I offer the thicker computer more money, that changes the amount of experience? Nope - it still has the correct betting behavior if it uses a probability of 0.5.
Consider the Sleeping Beauty problem with these computers - in the classic version, our person is asked to give their probability that they're in the possibility where there's one thin computer, or the world where there are two thin computers. The correct betting strategy is to bet as if you think the probability that there are two computers is 2/3 - weighting each computer equally.
Now, consider altering the experiment so that either there's one thin computer, or one double computer. We have two possibilities - either the correct betting probability is 1/2 and the computers seem to have equal "experience", or we bite the bullet and say that the correct betting probability is 2/3 for a double computer, 10/11 for a 10x thicker computer, 1000/1001 for a 1000x thicker computer, etc.
I think what's happening here is you're conflating different kinds of stakes in the betting game.
Will the computers be split apart at any point in the future? If so, will the reward stay with one of them, be divided between them, or will they each retain a separate instance the full reward?
In the case of money, clearly it is external to them. So they will have to divide it up somehow, if they split into multiple thin computers in the future - the utility is first divided by the thickness, restoring balance.
On the other hand, if you were giving them an ebook, say, or they were in a simulated environment and the reward was also within the environment (analogous to the many-worlds hypothesis), then yes, those odds are correct, since they can literally receive that many more rewards if they are the thick computer.
Who cares? In every case, the lightbulb gets changed, so the question is obviously meaningless!
or perhaps...
We can't conclude anything from the mere fact that the lightbulb was changed. The answer depends on your prior.
or even...
Jokes like this demonstrate the need for Anthropic Atheism Plus, a safe space where fallacies and know-nothing reductionism can be explored, free from malicious trolling.
and finally...
In order to finish the work of wrecking my own joke, here are some explanatory end-notes.
(1) The reference to Atheism Plus, a forum of progressives who split from the New Atheism movement, is a dig at nyan_sandwich's affiliation with neo-reaction.
(2) This whole "joke" came about because I thought your post and his post were not only stupid, but too stupid to be worth directly engaging.
(2a) For example, you seem to be saying that if two people give the same answer to a question, then there's only one person there.
(2b) Meanwhile, nyan_sandwich's rationale for eschewing anthropic reasoning is, "This reminds me way too much of souls... I don't believe in observers."
(3) In retrospect, the joke I should have made here was, "How many functionalists does it take to change a lightbulb?" (The point being that a functionalist perspective on lightbulb-changing would see no difference between one, two, or a hundred agents being responsible for it.) And I should have commented separately on the other post.
(4) Furthermore, perhaps I should concede that both posts are only half-stupid, and that the stupidity in question is learned stupidity rather than slack-jawed stupidity. Both posts do exhibit comprehension of some relatively complicated thought-experiments, even if the philosophy introduced in order to deal with them does contain some absolute howlers (see 2a, 2b, above).
(5) And of course, I'd better ostentatiously declare that I too am looking pretty foolish by this point. This is a perennial preemptive defense employed by mockers and jesters through the ages: yes, I was mean to you, but you don't need to be mean to me, for I shall be mean to myself. Yes, I admit, I too am a flawed human being. Mea culpa, mea maxima culpa, I will try to do better next time.
--metatroll, breaking character since January 2014
The point being that a functionalist perspective on lightbulb-changing would see no difference between one, two, or a hundred agents being responsible for it.
Only as long as the hundred people then go out for one ice cream cone afterwards.
Is behaviorism the right way to judge experience? Suppose you simply recorded the outcome of the sleeping beauty problem for a computer X, and then replayed the scenario a few times using the cached choice of X instead of actually running X again each time. For that matter, suppose you just accurately predict what X will conclude and never actually run X at all. Does X experience the same thing the same number of times in all these instances? I don't see a behavioral difference between running two thin computers layered on top of each other and using one thin computer and one cached/predicted result.
Another way to ask this is how thin can you slice the computer before it stops having experience? From a behaviorism perspective you can thin it all the way down to just a cached/predicted responses.
If we can't differentiate between how much experience a computer has when it's running thin or thick, then that implies we can't truly measure how much experience a computer has at different times or places. It's just an assumption that a computer now and a computer 5 minutes from now have the same measure of experience, but I don't see a way to test that assumption.
My intuitive guess is that all that matters is if the program is run. It doesn't matter if it's on a double-thick computer, or two computers, or what.
Quantum physics seems to suggest a different result. If you run a computer with twice the wave-form, it has four times the subjective probability. It could be that we got the equation wrong and all the values we're using are the square root of what they should be. It could be that there's discrete universes in there and the wave-form is just the square root of the density. It could be something I haven't thought of. But it could be really unexpected anthropics.
Subjective experience is also valued differently for humans - in some domains. Not all persons are equal in an airplane. You may pay extra if you are heavy - some may take this as unfair not allocating equal weight. Conversely tall and/or heavy people often experience a natural authority that comes with physical presence. Even if their live is consered equally worthy they nonetheless gain some advantage from their stature.
See also Gains from trade: Slug versus Galaxy - how much would I give up to control you?.
What do I claim explains this? The simple fact that you only offer the double computer one bet, not two
What makes you so confident that the double computer's offered one bet and the two separate computers are offered two? Because the two separate computers might make two different choices? Only if there's some other difference (e.g. you've fitted separate random number generators to the two separate computers, and the double computer only has one). Because you're paying out twice as much if you lose? Presumably you put twice as much current into the double computer as to each of the single computers. Because you, as the experimenter, are offering different experiences to the two separate computers? That just repeats the same problem one level higher (considering that there are probably Everett branches in which you took both different actions with the double computer).
Because if I put one dollar on each bet, I can only lose one dollar to the double computer, but can lose two dollars to the two computers.
The computer doesn't hold cash (clearly), it has account # and password of a bank account (or private key to bitcoin addresses if it's a particularly risky computer). The two thin computers therefore only have half as much money to bet. (Or they'll overdraw their account.)
How does your dollar get into the computers? Is there some dollar-input peripheral that the double computer only has one of and the two separate computers have one each? If so, isn't that the difference there?
Sure, one could specify it like that. But a dollar sliced down the middle is not legal tender.
My original intent was that I'd simply talk to the computer on an input/output peripheral and make a bet that way. Two peripherals would mean two bets.
But the difference isn't in the computers, it's in the peripherals. If you connect two peripherals to the two computers and one to the double computer then you can make two bets with the two computers and one with the double computer. If you connect one peripheral to the two computers and two peripherals to the double computer then you can make one bet with the two computers and two bets with the double computer. You haven't demonstrated any difference between the actual computers.
Think about the subjective probability assignments of the computers that's needed to generate correct betting behavior.
It's the same either way. The probability assignment might depend on how many dollar-inputs you connect, but it's the same whether you connect them to two computers or one double computer.
Ok, I'll do the math, then I'm tapping out.
The bet goes like this: you pay me X dollars, and if the coin is tails, I'll pay you $1. I offer the bet to each terminal.
At what X will the robot accept? Well, it will accept when ($1 - $X) * P(tails) - $X * P(heads) > 0. This simplifies to P(tails) > X.
So for example, if there's no duplication - I just flip a coin and offer the bet to a computer - the computer will accept for X < 0.5, reflecting a probability of 0.5.
If I duplicate the robot in the tails case, then it accepts the same bet (assuming complete selfishness), but now it does best if it accepts when X < 2/3, indicating P(tails) = 2/3.
If I attach an extra peripheral in the tails-case (and the computer doesn't notice), the bet is now: pay me X dollars if heads and 2X dollars if tails, for a payout of $2 if tails.
The robot will then accept if ($2 - $2X) * P(tails) - $X * P(heads) > 0. We can us that P(T) = 1-P(H) to expand this to 2/X > 1/P(tails)+1. Now if P(tails) = 1/2, the correct strategy is to accept when X<2/3.
In short, the probability assignment does not depend on how many peripherals the computer has, the different betting behavior follows straightforwardly from the payoffs of different betting strategies changing.
If you simulate someone on one thin computer connected to one dollar-input peripheral if heads, and one thin computer connected to one dollar-input peripheral if tails, and you offer them a bet on your coinflip, the correct probability is 1/2. If you ran one thin computer connected to one dollar-input peripheral if heads, and two thin computers connected to two dollar-input peripherals if tails, the correct probability is 2/3. If you ran one thin computer connected to one dollar-input peripheral if heads, and one thick computer connected to two dollar-input peripherals if tails, the correct probability is 2/3.
There's no difference between two thin computers and one thick computer - only a difference between two dollar-input peripherals or one dollar-input peripheral.
You can have same software running on multiple machines. There's still intermediate steps where you go from 1 to 2.
I think the question is fundamentally misguided with regards to the meaning of the word "probability" , and the phenomenalist position is the only correct one. I.e the question is meaningless. The "probability" is not a property of the world of sleeping beauty problem, it's just an intermediate value in the calculations, correctness of which is only defined by it's end use, e.g. in bets. Where the best value depends on which instances of the program we prefer to end up betting wrong.
See Cox's theorem for what kind of objects I'm talking about when I refer to a non-betting "probability."
I have a lot of sympathy for the idea that probability is just a helper for betting (i.e. decision theory), but what do you make of all the talk about limits of subjective frequencies, etc.? Do you think it's all meaningless, and the laws of physics that we observe to be "statistically true" (e.g. the second law of thermodynamics or the Born rule in QM) somehow come from betting as well?
Thermodynamics works fine under fully deterministic laws of physics. And with QM we don't have satisfactory quantum gravity so I wouldn't put much weight on implications of that.
Let's consider the sleeping beauty thing without abstracting out randomness. A coin is tossed, it bounces several times, making it final orientation highly sensitive to the original orientation. It comes to rest, heads up, or tails up (we don't know such things, ultimately because our head is a region of the universe smaller than the universe itself). For the given conditions it was physically impossible for it to have ended up the other side up. The betting is quite straightforward (dependent on whenever there's 2 bets vs 1 bet).
edit: and the probabilities of the cylinder landing on either side or the edge, they are properties of real world all right - they're the fraction of the hyper volume of possible initial phase space mapping to either final orientation, derivable without having to do statistical averaging.
If I'm running on a silicon computer, do I have twice as much subjective experience if my computer is twice as thick?
Why is this even a good question?
Consider a computer that was printed on a flat sheet. If we stick two of these computers (one a mirror image) together face to face, we get a thicker computer. And then if we peel them apart again, we get two thin computers! Suppose that we simulate a person using these computers. It makes sense that a person running on two thin computers has twice as much "experience" as a person running on just one (for example, in the Sleeping Beauty problem, the correct betting strategy is to bet as if the probability of making the bet in a given world is proportional to the number of thin computers). So if we take two people-computers and stick them together into one thicker person-computer, the thicker person contains twice as much "experience" as a thinner one - each of their halves has as much experience as a thin person, so they have twice as much experience.
Do I disagree? Well, I think it depends somewhat on how you cash out "experience." Consider the Sleeping Beauty problem with these computers - in the classic version, our person is asked to give their probability that they're in the possibility where there's one thin computer, or the world where there are two thin computers. The correct betting strategy is to bet as if you think the probability that there are two computers is 2/3 - weighting each computer equally.
Now, consider altering the experiment so that either there's one thin computer, or one double computer. We have two possibilities - either the correct betting probability is 1/2 and the computers seem to have equal "experience", or we bite the bullet and say that the correct betting probability is 2/3 for a double computer, 10/11 for a 10x thicker computer, 1000/1001 for a 1000x thicker computer, etc.
The bullet-biting scenario is equivalent to saying that the selfish desires of the twice-thick computer are twice as important. If one computer is one person, a double computer is then two people in a box.
But of course, if you have a box with two people in it, you can knock on the side and go "hey, how many of you people are in there? I'm putting in an order for chinese food, how many entrees should I get?" Instead, the double-thick computer is running exactly the same program as the thin computer, and will order exactly the same number of entrees. In particular, a double-thick computer will make evaluations of selfish vs. altruistic priorities exactly the same as a thin computer.
There is one exception to the previous paragraph - what if the computer is programmed to care about its own thickness, and measure it with external instruments since introspection won't do, and weight its desires more when it's thicker? This is certainly possible, but by putting the caring straight into the utility function, it removes any possibility that the caring is some mysterious "experience." It's just a term in the utility function - it doesn't have to be there, in fact by default it's not. Or, heck, your robot might as easily care more about things when the tides are high, that doesn't mean that high tides grant "experience."
The original Sleeping Beauty problem, now *that's* mysterious "experience." Ordinary computers enter, weighting the possibility by the number of computers leaves. So something happens when you merge the two computers into a double computer, to destroy that experience rather than conserving it.
What do I claim explains this? The simple fact that you only offer the double computer one bet, not two. Sure, the exact same signals go to the exact same wires in each case. Except for the prior information that says that the experimenter can only make 1 bet, not 2. In this sense, "experience" just comes from the ways in which our computer can interact with the world.
So since the a double-thick computer is not more selfish than a thin one (neglecting the tides), and will not expect to be a thick computer more often in the Sleeping Beauty problem, I'd say it doesn't have more "experience" than a thin computer.
EDIT: I use betting behavior as a proxy for probability here because it's easy to see which answer is correct. However, using betting behavior as a probability is not always valid - e.g. in the absent-minded driver problem. In the sleeping beauty case it only works because the payout structure is very simple. A safer way would be to derive the probabilities from the information available to the agents, which has been done elsewhere, but is harder to follow.