Are cryopreserved humans l-zombies?
keeping in mind that if they were an l-zombie, they would still say "I have conscious experiences, so clearly I can't be an l-zombie"?
As well they should. For l-zombies to do anything they need to be run, whereupon they stop being l-zombies.
At this point I would be mildly surprised (though pleased) to discover that there was any metaphysically significant "reality fluid," mostly because it seems that even in a universe without any everyone would have all of the same evidence for its existence that we do. "What the physical world actually looks like" seems likely to be a matter of preferences (perhaps combined with considerations of accessibility, though if you are willing to grant that inaccessible regions of the universe really exist, you probably are reduced to preferences).
If your preferences are sensitive to your own physical existence (for example if you don't care what happens whenever you don't actually exist) then there is a straightforward conflict of values between the two copies in the mugging (which can be resolved by bargaining in the usual way). If not then as you say its not clear why whether the decision-maker physically exists or not ought to have any significance to the problem.
My bet as to the best approach (and even my best bet as to what we will retrospectively regard as the best approach) is to accept a probable or at least plausible dependence of "really exist" on...
It seems at best fairly confused to say that an L-zombie is wrong because of something it would do if it were run, simply because we evaluated what it would say or do against the situation where it didn't. Where you keep saying "is" and "concludes" and "being" you should be saying "would", "would conclude", and "would be", all of which is a gloss for "would X if it were run", and in the (counter-factual) world where the L-zombie "would" do those things it "would be runnin...
Thank you for this post.
The concept of l-zombie is something that will be very helpful in allowing me to communicate my own beliefs about anthropics.
Why would somebody care about the properties of things that do not exist?
I think this is a real question, an interesting question, so don't downvote it because you think I am being flippant. Instead think about why you care about the properties of things that don't exist. I think anybody who cares about the properties of something that doesn't exist is equivocating on the meaning of doesn't exist: they are thinking something that doesn't exist is actually something, but with its "I exist" checkbox left unchecked. As opposed to the idea that ...
Actually, there probably aren't any p-zombies
Should that (from the first line of the third paragraph) be l-zombies?
If I've made a good moral choice and saved humanity I absolutely should feel happy about it - it means my feelings are aligned with my preferences (and even if the two were independent, other things being equal I'd rather be happy than sad).
I think the correct response to l-zombies is the same as that to p-zombies - that they really are impossible, and thus, contemplating their consequences is likely to lead us astray rather than yield valuable insights.
In general for counter-factual muggings, the way I like to think of it is that you would like to choose A, but as a rational agent you pre-committed to choosing B at a previous time point and are now physically incapable of making a different choice, thanks to an iron-clad pre-commitement originally intended to convince the omnipotent Omega that you were going to behave a certain way, by actually binding you to behave thusly.
But if you find that you can choose A, then by all means, choose A. It's evidence that you failed as a rational agent at a previous t...
If L-zombies have conscious experience (even when not being 'run'), does the concept even mean anything? Is there any difference, even in principle, between such an L-zombie and a 'real' person?
One can get kind of weird about whether or not a program is being run. After all, there exists a way to interpret the random motion of the molecules of a cup of hot tea as a succession of states corresponding to the execution of whatever program you want. Taking this line of argument to its ridiculous conclusion leads to Greg Egan's Dust Theory...
That seems right to me, but you have to be careful with the word "could" especially, when you are talking outside the realm of choices and probability.
I think a good way to think about it is through an AI. An AI that is currently running is like a person. If we have code for an AI that we decide never to run, that AI is an L-zombie.
Perhaps if you are an L-zombie and your decisions can influence the real world, then your decision doesn't matter. Omega has selected you from the space of all possible minds, which is unimaginably large, and asked for 1 bit of information, there will be a 50% chance that the bit will be either 0 or 1, since the space of all possible L-zombies has maximum entropy(?). Or perhaps not.
More generally, I think the idea that if you are an L-zombie your decisions might effect the real world is a violation of the thought experiment. It's presupposing something tha...
Good post, but...
I imagined myself as those L-zombies as I was reading through and trying to understand. Thus they're not L-zombies anymore. Did you do the same as you were writing?
Why stop at the written program level? What if you are about to type the final semi-colon in the description of a simulated human? When does it become an L-zombie or, alternatively, conscious? What about the day before you go to your office to finish the program? Maybe at the moment you made the decision to write this program? Where is the magical boundary? Is "finished program" just a convenient Schelling point?
Consider a Turing Machine whose input is the encoded state of the world as a binary integer, S, of maximum value 2^N-1, and which seeks SN positions into its binary tape and outputs the next N bits from the tape. Does that Turing Machine cause the same experience as an equivalent Turing Machine that simulates 10 seconds of the laws of physics for the world state S and outputs the resulting world state? I posit that the former TM actually causes no experience at all when run, despite the equivalence. So there probably exist l-zombies that would act as if...
The thought patterns of infants, people with traumatic brain injuries, people with some mental illnesses, people who are asleep and people on drugs can be as alien as you describe here. Non-human animal's thought patterns are also as alien as you like. Right here on this Earth are likely examples of what you describe. And in an infinite universe, yes for sure.
Reply to: Benja2010's Self-modification is the correct justification for updateless decision theory; Wei Dai's Late great filter is not bad news
"P-zombie" is short for "philosophical zombie", but here I'm going to re-interpret it as standing for "physical philosophical zombie", and contrast it to what I call an "l-zombie", for "logical philosophical zombie".
A p-zombie is an ordinary human body with an ordinary human brain that does all the usual things that human brains do, such as the things that cause us to move our mouths and say "I think, therefore I am", but that isn't conscious. (The usual consensus on LW is that p-zombies can't exist, but some philosophers disagree.) The notion of p-zombie accepts that human behavior is produced by physical, computable processes, but imagines that these physical processes don't produce conscious experience without some additional epiphenomenal factor.
An l-zombie is a human being that could have existed, but doesn't: a Turing machine which, if anybody ever ran it, would compute that human's thought processes (and its interactions with a simulated environment); that would, if anybody ever ran it, compute the human saying "I think, therefore I am"; but that never gets run, and therefore isn't conscious. (If it's conscious anyway, it's not an l-zombie by this definition.) The notion of l-zombie accepts that human behavior is produced by computable processes, but supposes that these computational processes don't produce conscious experience without being physically instantiated.
Actually, there probably aren't any l-zombies: The way the evidence is pointing, it seems like we probably live in a spatially infinite universe where every physically possible human brain is instantiated somewhere, although some are instantiated less frequently than others; and if that's not true, there are the "bubble universes" arising from cosmological inflation, the branches of many-worlds quantum mechanics, and Tegmark's "level IV" multiverse of all mathematical structures, all suggesting again that all possible human brains are in fact instantiated. But (a) I don't think that even with all that evidence, we can be overwhelmingly certain that all brains are instantiated; and, more importantly actually, (b) I think that thinking about l-zombies can yield some useful insights into how to think about worlds where all humans exist, but some of them have more measure ("magical reality fluid") than others.
So I ask: Suppose that we do indeed live in a world with l-zombies, where only some of all mathematically possible humans exist physically, and only those that do have conscious experiences. How should someone living in such a world reason about their experiences, and how should they make decisions — keeping in mind that if they were an l-zombie, they would still say "I have conscious experiences, so clearly I can't be an l-zombie"?
If we can't update on our experiences to conclude that someone having these experiences must exist in the physical world, then we must of course conclude that we are almost certainly l-zombies: After all, if the physical universe isn't combinatorially large, the vast majority of mathematically possible conscious human experiences are not instantiated. You might argue that the universe you live in seems to run on relatively simple physical rules, so it should have high prior probability; but we haven't really figured out the exact rules of our universe, and although what we understand seems compatible with the hypothesis that there are simple underlying rules, that's not really proof that there are such underlying rules, if "the real universe has simple rules, but we are l-zombies living in some random simulation with a hodgepodge of rules (that isn't actually ran)" has the same prior probability; and worse, if you don't have all we do know about these rules loaded into your brain right now, you can't really verify that they make sense, since there is some mathematically possible simulation whose initial state has you remember seeing evidence that such simple rules exist, even if they don't; and much worse still, even if there are such simple rules, what evidence do you have that if these rules were actually executed, they would produce you? Only the fact that you, like, exist, but we're asking what happens if we don't let you update on that.
I find myself quite unwilling to accept this conclusion that I shouldn't update, in the world we're talking about. I mean, I actually have conscious experiences. I, like, feel them and stuff! Yes, true, my slightly altered alter ego would reason the same way, and it would be wrong; but I'm right...
...and that actually seems to offer a way out of the conundrum: Suppose that I decide to update on my experience. Then so will my alter ego, the l-zombie. This leads to a lot of l-zombies concluding "I think, therefore I am", and being wrong, and a lot of actual people concluding "I think, therefore I am", and being right. All the thoughts that are actually consciously experienced are, in fact, correct. This doesn't seem like such a terrible outcome. Therefore, I'm willing to provisionally endorse the reasoning "I think, therefore I am", and to endorse updating on the fact that I have conscious experiences to draw inferences about physical reality — taking into account the simulation argument, of course, and conditioning on living in a small universe, which is all I'm discussing in this post.
NB. There's still something quite uncomfortable about the idea that all of my behavior, including the fact that I say "I think therefore I am", is explained by the mathematical process, but actually being conscious requires some extra magical reality fluid. So I still feel confused, and using the word l-zombie in analogy to p-zombie is a way of highlighting that. But this line of reasoning still feels like progress. FWIW.
But if that's how we justify believing that we physically exist, that has some implications for how we should decide what to do. The argument is that nothing very bad happens if the l-zombies wrongly conclude that they actually exist. Mostly, that also seems to be true if they act on that belief: mostly, what l-zombies do doesn't seem to influence what happens in the real world, so if only things that actually happen are morally important, it doesn't seem to matter what the l-zombies decide to do. But there are exceptions.
Consider the counterfactual mugging: Accurate and trustworthy Omega appears to you and explains that it just has thrown a very biased coin that had only a 1/1000 chance of landing heads. As it turns out, this coin has in fact landed heads, and now Omega is offering you a choice: It can either (A) create a Friendly AI or (B) destroy humanity. Which would you like? There is a catch, though: Before it threw the coin, Omega made a prediction about what you would do if the coin fell heads (and it was able to make a confident prediction about what you would choose). If the coin had fallen tails, it would have created an FAI if it has predicted that you'd choose (B), and it would have destroyed humanity if it has predicted that you would choose (A). (If it hadn't been able to make a confident prediction about what you would choose, it would just have destroyed humanity outright.)
There is a clear argument that, if you expect to find yourself in a situation like this in the future, you would want to self-modify into somebody who would choose (B), since this gives humanity a much larger chance of survival. Thus, a decision theory stable under self-modification would answer (B). But if you update on the fact that you consciously experience Omega telling you that the coin landed heads, (A) would seem to be the better choice!
One way of looking at this is that if the coin falls tails, the l-zombie that is told the coin landed heads still exists mathematically, and this l-zombie now has the power to influence what happens in the real world. If the argument for updating was that nothing bad happens even though the l-zombies get it wrong, well, that argument breaks here. The mathematical process that is your mind doesn't have any evidence about whether the coin landed heads or tails, because as a mathematical object it exists in both possible worlds, and it has to make a decision in both worlds, and that decision affects humanity's future in both worlds.
Back in 2010, I wrote a post arguing that yes, you would want to self-modify into something that would choose (B), but that that was the only reason why you'd want to choose (B). Here's a variation on the above scenario that illustrates the point I was trying to make back then: Suppose that Omega tells you that it actually threw its coin a million years ago, and if it had fallen tails, it would have turned Alpha Centauri purple. Now throughout your history, the argument goes, you would never have had any motive to self-modify into something that chooses (B) in this particular scenario, because you've always known that Alpha Centauri isn't, in fact, purple.
But this argument assumes that you know you're not a l-zombie; if the coin had in fact fallen tails, you wouldn't exist as a conscious being, but you'd still exist as a mathematical decision-making process, and that process would be able to influence the real world, so you-the-decision-process can't reason that "I think, therefore I am, therefore the coin must have fallen heads, therefore I should choose (A)." Partly because of this, I now accept choosing (B) as the (most likely to be) correct choice even in that case. (The rest of my change in opinion has to do with all ways of making my earlier intuition formal getting into trouble in decision problems where you can influence whether you're brought into existence, but that's a topic for another post.)
However, should you feel cheerful while you're announcing your choice of (B), since with high (prior) probability, you've just saved humanity? That would lead to an actual conscious being feeling cheerful if the coin has landed heads and humanity is going to be destroyed, and an l-zombie computing, but not actually experiencing, cheerfulness if the coin has landed heads and humanity is going to be saved. Nothing good comes out of feeling cheerful, not even alignment of a conscious' being's map with the physical territory. So I think the correct thing is to choose (B), and to be deeply sad about it.
You may be asking why I should care what the right probabilities to assign or the right feelings to have are, since these don't seem to play any role in making decisions; sometimes you make your decisions as if updating on your conscious experience, but sometimes you don't, and you always get the right answer if you don't update in the first place. Indeed, I expect that the "correct" design for an AI is to fundamentally use (more precisely: approximate) updateless decision theory (though I also expect that probabilities updated on the AI's sensory input will be useful for many intermediate computations), and "I compute, therefore I am"-style reasoning will play no fundamental role in the AI. And I think the same is true for humans' decisions — the correct way to act is given by updateless reasoning. But as a human, I find myself unsatisfied by not being able to have a picture of what the physical world probably looks like. I may not need one to figure out how I should act; I still want one, not for instrumental reasons, but because I want one. In a small universe where most mathematically possible humans are l-zombies, the argument in this post seems to give me a justification to say "I think, therefore I am, therefore probably I either live in a simulation or what I've learned about the laws of physics describes how the real world works (even though there are many l-zombies who are thinking similar thoughts but are wrong about them)."
And because of this, even though I disagree with my 2010 post, I also still disagree with Wei Dai's 2010 post arguing that a late Great Filter is good news, which my own 2010 post was trying to argue against. Wei argued that if Omega gave you a choice between (A) destroying the world now and (B) having Omega destroy the world a million years ago (so that you are never instantiated as a conscious being, though your choice as an l-zombie still influences the real world), then you would choose (A), to give humanity at least the time it's had so far. Wei concluded that this means that if you learned that the Great Filter is in our future, rather than our past, that must be good news, since if you could choose where to place the filter, you should place it in the future. I now agree with Wei that (A) is the right choice, but I don't think that you should be happy about it. And similarly, I don't think you should be happy about news that tells you that the Great Filter is later than you might have expected.