xlr8harder writes:

In general I don’t think an uploaded mind is you, but rather a copy. But one thought experiment makes me question this. A Ship of Theseus concept where individual neurons are replaced one at a time with a nanotechnological functional equivalent.

Are you still you?

Presumably the question xlr8harder cares about here isn't semantic question of how linguistic communities use the word "you", or predictions about how whole-brain emulation tech might change the way we use pronouns.

Rather, I assume xlr8harder cares about more substantive questions like:

  1. If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?
  2. Should I anticipate experiencing what my upload experiences?
  3. If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

My answers:

  1. Yeah.
  2. Yep.
  3. Yep, this is no big deal. A productive day for me might involve doing some work in the morning, getting a sandwich at Subway, destructively uploading my brain, then texting some friends to see if they'd like to catch a movie after I finish answering e-mails. ¯\_(ツ)_/¯

If there's an open question here about whether a high-fidelity emulation of me is "really me", this seems like it has to be a purely verbal question, and not something that I would care about at reflective equilibrium.

Or, to the extent that isn't true, I think that's a red flag that there's a cognitive illusion or confusion still at work. There isn't a special extra "me" thing separate from my brain-state, and my precise causal history isn't that important to my values.

I'd guess that this illusion comes from not fully internalizing reductionism and naturalism about the mind.

I find it pretty natural to think of my "self" as though it were a homunculus that lives in my brain, and "watches" my experiences in a Cartesian theater.

On this intuitive model, it makes sense to ask, separate from the experiences and the rest of the brain, where the homunculus is. (“OK, there’s an exact copy of my brain-state there, but where am I?”)

E.g., consider a teleporter that works by destroying your body, and creating an exact atomic copy of it elsewhere.

People often worry about whether they'll "really experience" the stuff their brain undergoes post-teleport, or whether a copy will experience it instead. "Should I anticipate 'waking up' on the other side of the teleporter? Or should I anticipate Oblivion, and it will be Someone Else who has those future experiences?"

This question doesn't really make sense from a naturalistic perspective, because there isn't any causal mechanism that could be responsible for the difference between "a version of me that exists at 3pm tomorrow, whose experiences I should anticipate experiencing" and "an exact physical copy of me that exists at 3pm tomorrow, whose experiences I shouldn't anticipate experiencing".

Imagine that the teleporter is located on Earth, and it sends you to a room on a space station that looks and feels identical to the room you started in. This means that until you exit the room and discover whether you're still on Earth, there's no way for you to tell whether the teleporter worked.

But more than that, there will be nothing about your brain that tracks whether or not the teleporter sent you somewhere (versus doing nothing).

There isn't an XML tag in the brain saying "this is a new brain, not the original"!

There isn't a Soul or Homunculus that exists in addition to the brain, that could be the causal mechanism distinguishing "a brain that is me" from "a brain that is not me". There's just the brain-state, with no remainder.

All of the same functional brain-states occur whether you enter the teleporter or not, at least until you exit the room. At every moment where the brain exists, the current state of the brain isn't affected by whether teleportation occurred.

So there isn't, within physics, any way for "the real you to be having an experience" in the case where the teleporter malfunctioned, and "someone else to be having the experience" in the case where the teleporter worked. (Unless this is a purely verbal distinction, unrelated to the three important-feeling questions we started with.)

Physics is local, and doesn't remember whether the teleportation occurred in the past.

Nor is there a law of physics saying "your subjective point of view immediately blips out of existence and is replaced by Someone Else's point of view if your spacetime coordinates change a lot in a short period of time (even though they don't blip out of existence when your spacetime coordinates change a little or change over a longer period of time)".

If that sort of difference can really and substantively change whether your experiences persist over time, it would have to be through some divine mechanism outside of physics.[1]


Why Humans Feel Like They Persist


Taking a step back, we can ask: what physical mechanism makes it feel as though I'm persisting over time? In normal cases, why do I feel so confident that I'm going to experience my future self's experiences, as opposed to being replaced by a doppelganger who will experience everything in my place?

Let's call "Rob at time 1" R1, "Rob at time 2" R2, and "Rob at time 3" R3.

R1 is hungry, and has the thought "I'll go to the fridge to get a sandwich". R2 walks to the fridge and opens the door. R3 takes a bite of the sandwich.

Question 1: Why is R2 bothering to open the fridge, even though it's R3 that will get to eat the sandwich? For that matter, why is R1 bothering to strategize about finding food, when it's not R1 who will realize the benefits?

Answer: Well, there's no need in principle for my time-slices to work together like that. Indeed, there are other cases where my time-slices work at cross purposes (like when I try to follow a diet but one of my time-slices says "no"). But it was reproductively advantageous for my ancestors' brains to generate and execute plans (including very fast, unconscious five-second plans), so they evolved to do so, rather than just executing a string of reflex actions.

Question 2: OK, but you could still achieve all that by having R1 think of R1, R2, and R3 as three different people. Rather than R1 thinking "I selfishly want a sandwich, so I'll go ahead and do multiple actions in sequence so that I get a sandwich", why doesn't R1 think "I altruistically want my friend R3 to have a sandwich, so I'll collaborate with R2 to do a favor for R3"?

Answer: Either of those ways of thinking would probably work fine in principle. Indeed, there's some individual and cultural variation in how much individual humans think of themselves as transtemporal "teams" versus persisting objects.

But it does seem like humans have a pretty strong inclination to think of themselves as psychologically persisting over time. I don't know why that is, but plausibly it has a lot to do with the general way humans think of objects: we say that a table is "the same table" even if it has changed a lot through years of usage. We even say that a caterpillar is "the same organism" as the butterfly it produces. We don't usually think of objects as a rapid succession of momentary blips, so it doesn't seem surprising that we think of our minds/brains as stable objects too, and use labels like "me" and "selfish" rather than "us" and "self-altruistic".

Question 3: OK, but it's not just that I'm using the arbitrary label "me" to refer to R1, R2, and R3. R1 anticipates experiencing the sandwich himself, and would anticipate this regardless of how he used language. Why's that?

Answer: Because R1 is being replaced by R2, an extremely similar brain that will likely remember the things R1 just thought. You're in a sense constantly passing the baton to a new person, as your brain changes over time. The feeling of being replaced by a new brain state that has around that much in common with your current brain state just is the experience that you're calling "persisting over time".

That experience of "persisting over time" isn't the experience of a magical Cartesian ghost that is observing a series of brain-states and acting as a single Subject for all of them. Rather, the experience of "persisting over time" just is the experience of each brain-states possessing certain kinds of information ("memories") about the previous brain-state in a sequence. (Along with R1, R2, and R3 having tons of overlapping personality traits, goals, etc.)

Some humans are more temporally unstable than others, and if a drug or psychotic episode interfered with your short-term memory enough, or caused your personality or values to change enough minute-to-minute, you might indeed feel as though "I'm the same person over time" has become less true.

(On the other hand, if you'd been born with that level of instability, it's less likely that you'd think there was anything weird about it. Humans can get used to a lot!)

There isn't a sharp black line in physics that determines how much a brain must resemble your own in order for you to "persist over time" into becoming that brain. There's just one brain-state that exists at one spacetime coordinate, and then another brain-state that exists at another spacetime coordinate.

If a brain-state A has quasi-sensory access to the experience of another brain-state B — if A feels like it "remembers" being in state B a fraction of a second ago — then A will typically feel as though it used to be B. If A doesn't have the same personality or values as B, then A will perhaps feel like they used to be B, but have suddenly changed into a very different sort of person.

Change enough, while still giving A immediate quasi-sensory access to B's state, and perhaps the connection will start to feel more dissociative or dreamlike; but there's no sharp line in physics to tell us how much change makes someone "no longer the same person".


Sleep and Film Reels

I find it easier to make sense of the teleporter scenario when I consider hypotheticals like "neuroscience discovers that you die and are reborn every night while you sleep", or "physics discovers that the entire universe is destroyed and an exact copy is recreated millions of times every second".

If we discovered one of those facts, would it make sense to freak out or go into mourning?

In that scenario, should we really start fretting about whether "I'm" going to "really experience" the thing that happens to my body five seconds from now, versus Someone Else experiencing it?

I think this would be pretty danged silly. You're right now experiencing what it's like to "toss the baton" from a past version of you to a future version of you, with zero consternation or anxiety, even though right now it's an open possibility that you're not "continuous".

Maybe the real, deep metaphysical Truth is that the universe is more like a film reel made up of many discrete frames (that feel continuous to us, because we're experiencing the frames from the inside, not looking at the reel from Outside The Universe), not something actually continuous.

I earnestly believe that the proper response to that hypothetical is: Who cares? For all I know, something like that could be true. But if it's true now, it was always true; I've been living that way my whole life. If the experiences I'm having as I write this sentence are the super scary Teleporter Death thing people keep saying I should worry about, then I already know what that's like, and it's chill.

If you aren't already bored by the whole topic (as you probably should be), you can play semantics and claim that I should instead say "the experiences we've been having as we write this sentence". Because this weird obscure discovery about metaphysics is somehow supposed to mean that in the world where we made this discovery, the Real Me is secretly constantly dying and being replaced...?

But whatever. If you're just redescribing the stuff I'm already experiencing and telling me that that's the scary thing, then I think you're too easily spooked by abstract redescriptions of ordinary life. Or if you're redescribing it but not trying to tell me I should freak out about your redescription, then it's just semantics, and I'll use pronouns in whichever way is most convenient.

Another way of thinking about this is: I am my brain, not a ghost or thing outside my brain. So if something makes no physical difference to my current brain-state, and makes no difference to any of my past or future brain-states, then I think it's just crazy talk to think that this metaphysical bonus thingie-outside-my-brain is the crucial thing that determines whether I exist, or whether I'm alive or dead, etc.

Thinking that my existence depends on some metaphysical "glue" outside of my brain, is like thinking that my existence depends on whether a magenta marble is currently orbiting Neptune. Why would the existence of some random Stuff out there in the cosmos that's not a Rob-time-slice brain-state, change how I should care about a Rob-time-slice brain-state, or change which brain-state (if any) I should anticipate?

Real life is more boring than the games we can play, striving to find a redescription of the mundane that makes the mundane sound spooky. Like children staring at campfire shadows and trying to will the shadows into looking like monsters.

Real life looks like going to bed at night and thinking about whether I want toast tomorrow morning, even though I don't know how sleep works and it's totally possible that sleep might involve shutting down my stream of consciousness at some point and then starting it up again.

Regardless of how a mature neuroscience of sleep ends up looking, I expect the me tomorrow to share a truly crazily extraordinarily massive number of memories, personality traits, goals, etc. in common with me.

I expect them to remember a ton of the things I do today, such that micro-decisions (like how I write this sentence) can influence a bunch of things about their state and their own future trajectory.

I can try to distract myself from those things with neurotic philosophy-101 ghost stories, but looking away from reality doesn't make it go away.


Weird-Futuristic-Technology Anxiety

Since there isn't a Soul that lives Outside The Film Reel and is being torn asunder from my brain-state by the succession of frames — there's just a bunch of brain-states — the anxiety about whether "I" should "really" anticipate any future experiences in Film Reel World is based in illusion.

But the only difference between this scenario and the teleporter one is that the teleporter scenario invokes a weird-sounding New Technology, whereas the sleep and Film Reel examples bake in "there's nothing new and weird happening, you've already been living your whole life this way". If you'd grown up using using teleporters all the time, then it would seem just as unremarkable as stepping through a doorway.

If a philosopher then came to you one day and said "but WHAT IF something KILLS YOU every time you step through a door and then a NEW YOU comes into existence on the other side!", you would just roll your eyes. If it makes no perceptible difference, then wtf are we even talking about?

And the same logic applies to mind uploading. There isn't some magical Extra Thing beyond the brain state, that could make it the case that one thing is You and another thing is Not You.

Sure, you're now made of silicon atoms rather than carbon atoms. But this is like discovering that Film Reel World alternates between one kind of metaphysical Stuff and another kind of Stuff every other second.

If you aren't worried about learning that the universe secretly metaphysically is in a state of Constant Oscillation between two types of (functionally indistinguishable) micro-particles, then why care about functionally irrelevant substrate changes at all?

(It's another matter entirely if you think carbon vs. silicon actually does make an inescapable functional, causal difference for which high-level thoughts and experiences your mind instantiates, and if you think that there's no way in principle to use a computer to emulate the causal behavior of a human mind. I think that's crazy talk, but it's crazy because of ordinary facts about physics / neuroscience / psych / CS, not because of any weird philosophical considerations.)


To Change Experience, You Have to Change Physics, Not Just Metaphysics

Scenario 1:

I step through a doorway.

At time 1, a brain is about to enter a doorway.

At time 2, an extremely similar brain is passing through the doorway.

At time 3, another extremely similar brain has finished passing through the doorway.

Scenario 2:

I step into a teleporter.

Here, again, there exist a series of extremely similar brain states before, during, and after I use the teleporter.

The particular brain states look no different in the teleporter case than if I'd stepped through a door; so if there's something that makes the post-teleporter Rob "not me" while also making the post-doorway Rob "me", then it must lie outside the brain states, a Cartesian Ghost.

Given all that, there's something genuinely weird about the fact that teleporters spook people more than walking through a door does.

It's like looking at a film strip, and being scared that if a blank slide were added in between every frame, this would somehow make a difference for the people living inside the movie. It's getting confused about the distinction between the physics of the movie's events and the meta-physics of "what the world runs on".

The same confusion can arise if we imagine flipping the order of all the frames in the film strip; or flipping the order of all the frames in the second half of the movie; or swapping the order of every pair of frames, like so:



From outside the movie, this can make the movie's events look more confusing or chaotic to us, the viewers. But if you imagine that the characters inside the movie would be the least bit bothered or confused by this rearrangement, you're making a clear mistake. To confuse the characters, you need to change what happens inside the frames, not just change the relationship between those frames.

I claim that a very similar cognitive hiccup is occurring when someone worries about their internal stream of consciousness halting due to a teleporter (and not halting due to stepping through a random doorway).

You're imagining that something about the context of the film cells — i.e., the stuff outside of the brain states themselves — is able to change your experiences.

But experiences just are brain things. To imagine that some of the unconscious goings-on in between two of your experiences can interfere with your Self is just the same kind of error as imagining that a movie character will be bothered, or will even subjectively notice, if you inject some empty frames into the movie while changing nothing else about the movie.


... And You Can't Change Experience With Just Any Old Change to Physics


As soon as a purple hat comes into existence on Pluto, my stream of consciousness will end and I will be imperceptibly replaced by an exact copy of myself that is experiencing a different stream of consciousness.

This exact copy of me will be physically identical to me in every respect, and will have all of my memories, personality traits, etc. But they won't be me. The hat, if such a hat ever comes into being, will kill me.

What, specifically, is wrong with this claim?

Well, one thing that's wrong with the claim is that Pluto is very far away from the Earth.

But the idea of a hat ending my existence seems very strange even if the hat is in closer proximity to me. Even putting a hat on my head seems like it shouldn't be enough to end my stream of consciousness, unless there's something special about the hat that will actually drastically change my brain-state. (E.g., maybe the hat is wired up with explosives.)

The point of this example being:

You can call the Ghost a "Soul", and make it obvious that we're invoking magic.

Or you can call it a "special kind of causal relationship (that's able to preserve selfhood)", and make it sound superficially scientific. (Or at least science-compatible.)

You can hypothesize that there's something special about the causal process that produces new brain-states in the "walk through a doorway" case — something "in the causality itself" that makes the post-doorway self me and the post-teleporter self not me.

But of course, this "causal relationship" is not a part of the brain state. Reify causality all you want; the issue remains that you're positing something outside the brain, outside you and your experiences, that is able to change which experiences you should anticipate without changing any of the experiences or brain-states themselves.

The brain states exist too, whatever causal relationships they exhibit. To say that exactly the same brain states can exist, and yet something outside of those states is changing a perceptible feature of those experiences ("which experience comes next in this subjective flow that's being experienced; what I should expect to see next"), without changing any of the actual brain states, is just as silly whether that something is a "causal relationship" or a purple hat.

This principle is easier to motivate in the case of the hat, because hats are a lot more concrete, familiar, and easy to think about than some fancy philosophical abstraction like "causal relationship". But the principle generalizes; random objects and processes out there, whether fancy-sounding or perfectly mundane, can't perceptibly change my experience (unless they change which brain states occur).

Likewise, it's easier to see that something on Pluto can't suddenly end my stream of consciousness, than to see that something physically (or metaphysically?) "nearby" can't suddenly end my stream of consciousness (without leaving a mess). But the principle generalizes; being nearby or connected to something doesn't open the door to arbitrary magical changes, absent some mechanism for how that exact change is caused by that exact physical process.

If we were just talking about word definitions and nothing else, then sure, define "self" however you want. You have the universe's permission to define yourself into dying as often or as rarely as you'd like, if word definitions alone are what concerns you.

But this post hasn't been talking about word definitions. It's been talking about substantive predictive questions like "What's the very next thing I'm going to see? The other side of the teleporter? Or nothing at all?"

There should be an actual answer to this, at least to the same degree there's an answer to "When I step through this doorway, will I have another experience? And if so, what will that experience be?"

And once we have an answer, this should change how excited we are about things like mind uploading. If my stream of consciousness is going to end with my biological death no matter what I do, then mind uploading sounds a lot less exciting!

Or, equivalently: If my experiences were a matter of "displaying images for a Cartesian Homunculus", and the death of certain cells in the brain severs the connection between my brain and the Homunculus, then there's no obvious reason I should expect this exact same Homunculus to establish a connection to an uploaded copy of my brain.

It's only if I'm in my brain, just an ordinary part of physics, that mind uploading makes sense as a way to extend my lifespan.

Causal relationships and processes obviously matter for what experiences occur. But they matter because they change the brain-states themselves. They don't cause additional changes to experience beyond the changes exhibited in the brain.


Having More Than One Future

I've tried to keep this post pretty simple and focused. E.g., I haven't gone into questions like "What happens if you make two uploads of me? Which one should I anticipate having the experiences of?"

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

If you've already bitten the bullet on things like the teleporter example, then I don't think this should actually be particularly counter-intuitive. If one copy of my brain exists at time 1 (Rob-x), and two almost-identical copies of my brain (Rob-y and Rob-z) exist at time 2, then there's going to be a version of me that's Rob-y, and a version of me that's Rob-z, and each will have equal claim to being "the next thing I experience".

In a world without magical Cartesian Homunculi, this has to be how things work; there isn't any physical difference between Rob-y and Rob-z that makes one of them my True Heir and the other a False Pretender. They're both just future versions of me.

"You should anticipate having both experiences" sounds sort of paradoxical or magical, but I think this stems from a verbal confusion. "Anticipate having both experiences" is ambiguous between two scenarios:

  • Scenario 1: "Split-screen mode." My stream of consciousness continues, but it somehow magically splits into a portion that's Rob-y and a different portion that's Rob-z, as though the Cartesian Homunculus were trying to keep an eye on both brains at once.
  • Scenario 2: "Two separate screens." My stream of consciousness continues from Rob-x to Rob-y, and it also continues from Rob-x to Rob-z. Or, equivalently: Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels exactly as though he was just Rob-x (since each of these slightly different people has all the memories, personality traits, etc. of Rob-x — just as though they'd stepped through a doorway).

Scenario 1 is crazy talk, and it's not the scenario I'm talking about. When I say "You should anticipate having both experiences", I mean it in the sense of Scenario 2.

Scenario 2 is pretty unfamiliar to us, because we don't currently live in a world where we can readily copy-paste our own brains. And accordingly, it's a bit awkward to talk about Scenario 2; the English language is adapted to a world where "humans don't fork" has always been a safe assumption.

But there isn't a mystery about what happens. If you think there's something mysterious or unknown about what happens when you make two copies of yourself, then I pose the question to you:

What concrete fact about the physical world do you think you're missing? What are you ignorant of?

Alternatively, if you're not ignorant of anything, then: how can there be a mystery here? (Versus just "a weird way the world can sometimes end up".)



  1. ^

    And insofar as it's your physical brain thinking these thoughts right now, unaltered by any divine revelation, it would have to be a coincidence that this "I would blip out of existence in case A but not case B" hunch is correct. Because the reason your brain has that intuition is a product of the brain's physical, causal history, and is not the result of you making any observation that's Bayesian evidence for this mechanism existing.

    Your brain is not causally entangled with any mechanism like that; you'd be thinking the same thoughts whether the mechanism existed or not. So while it's possible that you're having this hunch for reasons unrelated to the hunch being correct, and yet the hunch be correct anyway, you shouldn't on reflection believe your own hunch. Any Bayesian evidence for this hypothesis would need to come from some source other than the hunch/intuition.

New Comment
117 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

I think there's a pretty strong argument to be more wary about uploading. It's been stated a few times on LW, originally by Wei Dai if I remember right, but maybe worth restating here.

Imagine the uploading goes according to plan, the map of your neurons and connections has been copied into a computer, and simulating it leads to a person who talks, walks in a simulated world, and answers questions about their consciousness. But imagine also that the upload is being run on a computer that can apply optimizations on the fly. For example, it could watch the input-output behavior of some NN fragment, learn a smaller and faster NN fragment with the same input-output behavior, and substitute it for the original. Or it could skip executing branches that don't make a difference to behavior at a given time.

Where do we draw the line which optimizations to allow? It seems we cannot allow all behavior-preserving optimizations, because that might lead to a kind of LLM that dutifully says "I'm conscious" without actually being so. (The p-zombie argument doesn't apply here, because there is indeed a causal chain from human consciousness to an LLM saying "I'm conscious" - which goes through the LLM... (read more)

Yeah, at some point we'll need a proper theory of consciousness regardless, since many humans will want to radically self-improve and it's important to know which cognitive enhancements preserve consciousness.

Yeah. My point was, we can't even be sure which behavior-preserving optimizations (of the kind done by optimizing compilers, say) will preserve consciousness. It's worrying because these optimizations can happen innocuously, e.g. when your upload gets migrated to a newer CPU with fancier heuristics. And yeah, when self-modification comes into the picture, it gets even worse.

[Epistemic status: napkin] My current-favourite frame on "qualia" is that it refers to the class of objects we can think about (eg, they're part of what generates what I say rn) for which behaviour is invariant across structure-preserving transformations. (There's probably some cool way to say that with category theory or transformations, and it may or may not give clarity, but idk.) Eg, my "yellow" could map to blue, and "blue" to yellow, and we could still talk together without noticing anything amiss even if your "yellow" mapped to yellow for you. Both blue and yellow are representational objects, the things we use to represent/refer to other things with, like memory-addresses in a machine.  For externally observable behaviour, it just matters what they dereference to, regardless of where in memory you put them.  If you swap two representational objects, while ensuring you don't change anything about how your neurons link up to causal nodes outside the system, your behaviour stays the same. Note that this isn't the case for most objects.  I can't swap hand⇄tomato, without obvious glitches like me saying "what a tasty-looking tomato!" and trying to eat my hand.  Hands and tomatoes do not commute. It's what allows us to (try to) talk about "tomato" as opposed to just tomato, and explains why we get so confused when we try to ground out (in terms of agreed-upon observables) what we're talking about when we talk about "tomato". But how/why do we have representations for our representational objects in the first place?  It's like declaring a var (address₁↦value), and then declaring a var for that var (address₂↦address₁) while being confused about why the second dereferences to something 'arbitrary'. Maybe it starts when somebody asks you "what do you mean by 'X'?", and now you have to map the internal generators of [you saying "X"] in order to satisfy their question.  Or not.  Probably not.  Napkin out.
We can use the same thought experiments that Chalmers uses to establish a fine-grain-functionally-isomorphic copy had the same qualia, modify them and show that anything that acts like us has our qualia. The LLM character (rather than the LLM itself) will be conscious to the extent to which its behavior is I/O identical to the person. Edit: Oh, sorry, this is an old comment. I got this recommended... somehow... Edit2: Oh, it was curated yesterday.
Well, a thing that acts like us in one particular situation (say, a thing that types "I'm conscious" in chat) clearly doesn't always have our qualia. Maybe you could say that a thing that acts like us in all possible situations must have our qualia? This is philosophically interesting! It makes a factual question (does the thing have qualia right now?) logically depend on a huge bundle of counterfactuals, most of which might never be realized. What if, during uploading, we insert a bug that changes our behavior in one of these counterfactuals - but then the upload never actually runs into that situation in the course of its life - does the upload still have the same qualia as the original person, in situations that do get realized? What if we insert quite many such bugs? Moreover, what if we change the situations themselves? We can put the upload in circumstances that lead to more generic and less informative behavior: for example, give the upload a life where they're never asked to remember a particular childhood experience. Or just a short life, where they're never asked about anything much. Let's say the machine doing the uploading is aware of that, and allowed to optimize out parts that the person won't get to use. If there's a thought that you sometimes think, but it doesn't influence your I/O behavior, it can get optimized away; or if it has only a small influence on your behavior, a few bits' worth let's say, then it can be replaced with another thought that would cause the same few-bits effect. There's a whole spectrum of questionable things that people tend to ignore when they say "copy the neurons", "copy the I/O behavior" and stuff like that.
Such optimizations are a reason I believe we are not in a simulation. Optimizations are essential for a large sim. I expect them not to be consciousness preserving
1Joseph Miller
Surely 'you' are the algorithm, not the implementation. If I get refactored into a giant lookup table, I don't think that makes the algorithm any less 'me'.

I find myself strongly disagreeing with what is being said in your post. Let me preface by saying that I'm mostly agnostic with respect to the possible "explanations" of consciousness etc,  but I think I fall squarely within camp 2. I say mostly because I  lean moderately towards physicalism.

First, an attempt to describe my model of your ontology:

You implicitly assume that consciousness / subjective experience can be reduced to a physical description of the brain, which presumably you model as a classical (as opposed to quantum) biological electronic circuit.  Physically, to specify some "brain-state" (which I assume is essentially the equivalent of a "software snapshot" in a classical computer) you just need to specify a circuit connectivity for the brain, along with the currents and voltages between the various parts of the circuit (between the neurons let's say). This would track with your mentions of reductionism and physicalism and the general "vibe" of your arguments.  In this case I assume you treat conscious experience roughly as "what it feels like" to be software that is self-referential on top of taking in external stimuli from sensors. This software ... (read more)

9Rob Bensinger
There are always going to be many different ways someone could object to a view. If you were a Christian, you'd perhaps be objecting that the existence of incorporeal God-given Souls is the real crux of the matter, and if I were intellectually honest I'd be devoting the first half of the post to arguing against the Christian Soul. Rather than trying to anticipate these objections, I'd rather just hear them stated out loud by their proponents and then hash them out in the comments. This also makes the post less boring for the sorts of people who are most likely to be on LW: physicalists and their ilk. Why do you assume that you wouldn't experience the copy's version of events? The un-copied version of you experiences walking into the room, sitting there, hearing the scanner working, and hearing it stop; then that version of you experiences walking out. It seems like nothing special happened in this procedure; this version of you doesn't feel anything weird, and doesn't feel like their "consciousness split into two" or anything. The copied version of you experiences walking into the room, sitting here, hearing the scanner working, and then an instantaneous experience of (let's say) feeling like you've been teleported into another room -- you're now inside the simulation. Assuming the simulation feels like a normal room, it could well seem like nothing special happened in this procedure -- it may feel like blinking and seeing the room suddenly change during the blink, while you yourself remain unchanged. This version of you doesn't necessarily feel anything weird either, and they don't feel like their "consciousness split into two" or anything. It's a bit weird that there are two futures, here, but only one past -- that the first part of the story is the same for both versions of you. But so it goes; that just comes with the territory of copying people. If you disagree with anything I've said above, what do you disagree with? And, again, what do you mean by sayin

First off, would you agree with my model of your beliefs? Would you consider it an accurate description?

Also, let me make clear that I don't believe in cartesian souls. I, like you, lean towards physicalism, I just don't commit to the explanation of consciousness  based on the idea of the brain as a **classical** electronic circuit. I don't fully dismiss it either, but I think it is worse on philosophical grounds than assuming that there is some (potentially minor) quantum effect going on inside the brain that is an integral part of the explanation for our conscious experience. However, even this doesn't feel fully satisfying to me and this is why I say that I am agnostic. When responding to my points, you can assume that I am a physicalist, in the sense that I believe consciousness can probably be described using physical laws, with the added belief that these laws **may** not be fully understandable by humans.  I mean this in the same way that a cat for example would not be able to understand the mechanism giving rise to consciousness, even if that mechanism turned out to be based on the laws of classical physics (for example if you can just explain consciousness as sof... (read more)

6Rob Bensinger
Why would the laws of physics conspire to vindicate a random human intuition that arose for unrelated reasons? We do agree that the intuition arose for unrelated reasons, right? There's nothing in our evolutionary history, and no empirical observation, that causally connects the mechanism you're positing and the widespread human hunch "you can't copy me". If the intuition is right, we agree that it's only right by coincidence. So why are we desperately searching for ways to try to make the intuition right? Why is this an advantage of a theory? Are you under the misapprehension that "hypothesis H allows humans to hold on to assumption A" is a Bayesian update in favor of H even when we already know that humans had no reason to believe A? This is another case where your theory seems to require that we only be coincidentally correct about A ("sufficiently complex arrangements of water pipes can't ever be conscious"), if we're correct about A at all. One way to rescue this argument is by adding in an anthropic claim, like: "If water pipes could be conscious, then nearly all conscious minds would be instantiated in random dust clouds and the like, not in biological brains. So given that we're not Boltzmann brains briefly coalescing from space dust, we should update that giant clouds of space dust can't be conscious." But is this argument actually correct? There's an awful lot of complex machinery in a human brain. (And the same anthropic argument seems to suggest that some of the human-specific machinery is essential, else we'd expect to be some far-more-numerous observer, like an insect.) Is it actually that common for a random brew of space dust to coalesce into exactly the right shape, even briefly?

You're missing the bigger picture and pattern-matching in the wrong direction. I am not saying the above because I have a need to preserve my "soul" due to misguided intuitions. On the contrary, the reason for my disagreement is that I believe you are not staring into the abyss of physicalism hard enough. When I said I'm agnostic in my previous comment, I said it because physics and empiricism lead me to consider reality as more "unfamiliar"  than you do (assuming that my model of your beliefs is accurate). From my perspective, your post and your conclusions are written with an unwarranted degree of certainty, because imo your conception of physics and physicalism is too limited. Your post makes it seem like your conclusions are obvious because "physics" makes them the only option, but they are actually a product of implicit and unacknowledged philosophical assumptions, which (imo) you inherited from intuitions based on classical physics. By this I mean the following:

It seems to me that when you think about physics, you are modeling reality (I intentionally avoid the word "universe" because it evokes specific mental imagery) as a "scene" with "things" in it. You mentally take ... (read more)

So in reading your comments of this post, I feel like I am reading comments made by a clone of my own mind. Though you articulate my views better than I can. This particular comment you make, I don't think it gets The attention it deserves. It was pretty revolutionary for myself when I learned to think of almost every worldview at a model of reality. It's most revolutionary when one realizes what is arguably an outdated Newtonian view to fall into this category of model. It really highlights that actual reality is at the least very hard to get at. This is a severe an issue with regards to consciousness.
  Are you trying to say that quantum physics provides evidence that physical reality is subjective, with conscious observers having a fundamental role? Rob implicitly assumes the position advocated by The Quantum Physics Sequence, which argues that reality exists independently of observers and that quantum stuff doesn't suggest otherwise. It's just one of the many presuppositions he makes that's commonly shared on here. If that's your main objection, you should make that clear.
I would say that it is irrelevant for the points the post/Rob is trying to make whether consciousness is classical or quantum, given that conscious experience has, AFAIK, never been reported to be 'quantum' (i.e. that we don't seem to experience superpositions or entanglement) and that we already have straightforward classical examples of lack of conscious continuity (namely: sleeping). In the case of sleeping and waking up it is already clear that the currently awake consciousness is modeling its relation to past consciousnesses in that body through memories alone. Even without teleporters, copiers, or other universes coming into play, this connection is very fragile. How sure can a consciousness be that it is the same as the day before or as one during lucid parts of dreams? If you add brain malfunctions such as psychoses or dissociative drugs such as ketamine to the mix, the illusion of conscious continuity can disappear completely quite easily. I like to word it like this: A consciousness only ever experiences what the brain that produces it can physically sense or synthesize. With that as a starting point, modeling what will happen in the various thought experiments and analyses of conscious experience becomes something like this: "Given that there is a brain there, it will produce a consciousness, which will remember what is encoded in the structure of that brain and which will experience what that brain senses and synthesizes in that moment." There is no assumption that consciousness is classical in that, I believe. There is also no assumption of continuity in that, which I think is important as in my opinion that assumption is quite shaky and misdirects many discussions on consciousness. I'd say that the value in the post is in challenging that assumption.
The currently awake consciousness is located in the brain, which has physical continuity with its previous states. You don't wake up as a different person because "you" are the brain (possibly also the rest of the body depending how it affects cognition but IDK) and the brain does not cease to function when you fall asleep.
I agree on the physical continuity of the brain, but I don't think this transfers to continuity of the consciousness or its experience. It is defining "you" as that physical brain, rather than the conscious experience itself. It's like saying that two waves are the same because they are produced by the same body of water. Imagine significant modifications to your brain while you are asleep in such a way that your memories are vastly different, so much as to represent another person. Would the consciousness that is created on waking up experience a connection to the consciousness that that brain produced the day(s) before or to the manufactured identity? Even you, now, without modifications, can't say with certainty that your 'yesterday self' was experienced by the same consciousness as you are now (in the sense of identity of the conscious experience). It feels that way as you have memories of those experiences, but it may have been experienced by 'someone else' entirely. You have no way of discerning that difference (nor does anyone else).
The conscious experience is not extricable from the physical brain; it has your personality because the personality that you are is the sum total of everything in your brain. The identity comes from the brain; if it were somehow possible to separate consciousness from the rest of the mind, that consciousness wouldn't still be you, because you're the entire mind. I would... not consider the sort of brain modification you're describing to preserve physical continuity in the relevant sense? It sounds like it would, to create the described effects, involve significant alterations in portions of the brain wherein (so to speak) your identity is stored, which is not what normally happens when people sleep.
I think we are in agreement that the consciousness is tied to the brain. Claiming equivalency is not warranted, though: The brain of a dead person (very probably, I'm sure you'd agree) contains no consciousness. Let's not dwell on this, though: I am definitely not claiming that consciousness exists outside of the brain, just that asserting physical continuity of the brain is not enough by itself to show continuity of conscious experience. With regard to the modifications: Your line of reasoning runs into the classic issues of philosophical identity, as shown by the Ship of Theseus thought experiment or simpler yet, the Sorites paradox. We can hypothesize every amount of alterations from just modifying one atom to replacing the entire brain. Given your position, you'd be forced to choose an arbitrary amount of modifications that breaks the continuity and somehow changes consciousness A-modified-somewhat into consciousness B (or stated otherwise: from 'you waking up a somewhat changed person' to 'someone else waking up in your body'). Approaching conscious experience without the assumption of continuity but from the moment it exists in does not run into this problem.
(Assuming a frame of materialism, physicalism, empiricism throughout even if not explicitly stated) Some of your scenarios that you're describing as objectionable would reasonably be described as emulation in an environment that you would probably find disagreeable even within the framework of this post.  Being emulated by a contraption of pipes and valves that's worse in every way than my current wetware is, yeah, disagreeable even if it's kinda me.  Making my hardware less reliable is bad.  Making me think slower is bad.  Making it easier for others to tamper with my sensors is bad.  All of these things are bad even if the computation faithfully represents me otherwise. I'm mostly in the same camp as Rob here, but there's plenty left to worry about in these scenarios even if you don't think brain-quantum-special-sauce (or even weirder new physics) is going to make people-copying fundamentally impossible.  Being an upload of you that now needs to worry about being paused at any time or having false sensory input supplied is objectively a worse position to be in in. The evidence does seem to lean in the direction that non-classical effects in the brain are unlikely, neurons are just too big for quantum effects between neurons, and even if there were quantum effects within neurons, it's hard to imagine them being stable for even as long as a single train of thought.  The copy losing their train of thought and having momentary confusion doesn't seem to reach the bar where they don't count as the same person?  And yet weirder new physics mostly requires experiments we haven't thought to do yet, or experiments is regimes we've not yet been able to test.  Whereas the behavior of things at STP in water is about as central to things-Science-has-pinned-down as you're going to get.   You seem to hold that the universe maybe still has a lot of important surprises in store, even within the central subject matter of century old fields?  Do you have any kind of intuition pum

If you think there’s something mysterious or unknown about what happens when you make two copies of yourself

Eliezer talked about some puzzles related to copying and anticipation in The Anthropic Trilemma that still seem quite mysterious to me. See also my comment on that post.

the English language is adapted to a world where "humans don't fork" has always been a safe assumption.

If we can clone ourselves, language would probably quickly follow. The bigger change would probably be the one about social reality. What does it mean to make a promise? Who is the entity you make a trade with? Is it the collective of all the yous? Only one? But which one if they split? The yous resulting from one origin will presumably have to share or split their resources. How will they feel about it? Will they compete or agree? If they agree it makes more sense for them to feel more like a distributed being. The thinking of "I" might get replaced by an "us".

Reproduction and evolution is arguably literally how biology forks. You can also think of it as a tree of consciousness. Cloning would probably work the same in regards to consciousness. My clone would be a distinct branch of consciousness. There's little reason to think thought experiment level precision of cloning/duplication and teleportation are at all physically possible though.
It seems like the way we talk about the results of a coin flip would be a good start for how we'd talk about being cloned, although it's rare for a coin flip to have such a massive impact on our life after that point

What does it mean when one "should anticipate" something? At least in my mind, it points strongly to a certain intuition, but the idea behind that intuition feels confused. "Should" in order to achieve a certain end? To meet some criterion? To boost a term in your utility function?

I think the confusion here might be important, because replacing "should anticipate" with a less ambiguous "should" seems to make the problem easier to reason about, and supports your point.

For instance, suppose that you're going to get your brain copied next week. After you get copied, you'll take a physics test, and your copy will take a chemistry test (maybe this is your school's solution to a scheduling conflict during finals). You want both test scores to be high, but you expect taking either test without preparation will result in a low score. Which test should you prepare for?

It seems clear to me that you should prepare for both the chemistry test and the physics test. The version of you that got copied will be able to use the results of the physics preparation, and the copy will be able to use the copied results of the chemistry preparation. Does that mean you should anticipate taking a chemistry test and anticipate taking a physics test? I feel like it does, but the intuition behind the original sense of "should anticipate" seems to squirm out from under it.

I have a closely related objection/clarification.  I agree with the main thrust of Rob's post, but this part: ..strikes me as confused or at least confusing. Take your chemistry/physics tests example.  What does "I anticipate the experience of a sense of accomplishment in answering the chemistry test" mean?  Well for one thing, it certainly indicates that you believe the experience is likely to happen (to someone).  For another, it often means that you believe it will happen to you - but that invites the semantic question that Rob says this isn't about.  For a third - and I propose that this is a key point that makes us feel there is a "substantive" question here - it indicates that you empathize with this future person who does well on the test. But I don't see how empathizing or not-empathizing can be assessed for accuracy.  It can be consistent or inconsistent with the things one cares about, which I suppose makes it subject to rational evaluation, but that looks different from accuracy/inaccuracy.
4Rob Bensinger
In the OP: "Should" in order to have more accurate beliefs/expectations. E.g., I should anticipate (with high probability) that the Sun will rise tomorrow in my part of the world, rather than it remaining night.

Suppose someone draws a "personal identity" line to exclude this future sunrise-witnessing person.  Then if you claim that, by not anticipating, they are degrading the accuracy of the sunrise-witness's beliefs, they might reply that you are begging the question.

Here's a thought experiment.

In version A, I have a button that non invasively scans my brain and creates 10 perfect copies of my brain state in a computer. I press the button. For an instant, 11 identical mind states exist in the universe. Then each mind starts diverging along different causal chains.

Intuitively, I expect the following:

  • I won't experience anything unusual after pressing the button (eg, I won't wake up in a computer). I will still feel that I am in my physical body, in the room with the button
  • each of the mind copies will feel that they ar
... (read more)

I’d guess that this illusion comes from not fully internalizing reductionism and naturalism about the mind.

Naturalism and reductionism are not sufficient to rigourously prove either form of computationalism -- that performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual.

This has been going on for years: most rationalists believe in computationalism, none have a really good reason to.

Arguing down Cartesian dualism (the thing rationalists a... (read more)

2Rob Bensinger
Sure; from my perspective, you're saying the same thing as me. How am I supposed to choose between them?
By "equally" I meant: "in the same ways (and to the same degree)". If you actually believe in florid many worlds, you would end up pretty insuoucient, since everything possible happens, and nothing can be avoided.

An interesting consequence of your description is that resurrection is possible if you can manage to reconstruct the last brain state of someone who had died.  If you go one one step further, then I think it is fairly likely that experience is eternal, since you don't experience any of the intervening time (akin to your film reel analogy with adding extra frames in between) being dead and since there is no limit to how much intervening time can pass.

*preferably not the last state but some where the person felt normal. I believe that's right! Though, if person can be reconstructed from N bits of information, and dead body retains K << N, then we need to save N-K bits (or maybe all N, for robustness) somewhere else. It's an interesting question how many bits can be inferred from social networks trace of the person, actually.
Well ultimately no information about the past is truly lost as far as we know.  A hyper-advanced civilization could collect all the thermal radiation from earth reflected off of various celestial bodies and recover a near complete history, at least in principle.  So I think the more you make it easy for yourself to be reconstructed/resurrected/what have you the sooner it would likely be, and the less alien of an environment you would find yourself in after the fact.  Cryo is a good example of having a reasonable expectation of where to end up barring catastrophe since you are preserving a lot of you in good form.

Loved the post and all the comments <3

Here is I think an interesting scenario / thought experiment:

  1.  A copy of a person is made while that original person is sleeping on a bed.
  2. The original person is moved to a sofa while still sleeping.
  3. The copy (which is also sleeping) is put in the bed at the exact same position where the original person was.
  4. After a while the original and the copy both wake up and can see each other (we assume they are both completely oblivious to exactly what happened while they were sleeping and that they didn't dream or they dre
... (read more)

Wouldn't it follow that in the same way you anticipate the future experiences of the brain that you "find yourself in" (i.e. the person reading this) you should anticipate all experiences, i.e. that all brain states occur with the same kind of me-ness/vivid immediacy?

It seems that since there is nothing further than makes the experiences (that are these brains states, in this body that is writing these sentences) in some way special so that they're "mine" (there is no additional "me-ghost"), then those particular brain states aren't any different from all ... (read more)

3Rob Bensinger
What's the empirical or physical content of this belief? I worry that this may be another case of the Cartesian Ghost rearing its ugly head. We notice that there's no physical thingie that makes the Ghost more connected to one experience or the other; so rather than exorcising the Ghost entirely, we imagine that the Ghost is connected to every experience simultaneously. But in fact there is no Ghost. There's just a bunch of experience-moments implemented in brain-moments. Some of those brain-moments resemble other brain-moments, either by coincidence or because of some (direct or indirect) causal link between the brain-moments. When we talk about Brain-1 "anticipating" or "becoming" a future brain-state Brain-2, we normally mean things like: * There's a lawful physical connection between Brain-1 and Brain-2, such that the choices and experiences of Brain-1 influence the state of Brain-2 in a bunch of specific ways. * Brain-2 retains ~all of the memories, personality traits, goals, etc. of Brain-1. * If Brain-2 is a direct successor to Brain-1, then typically Brain-2 can remember a bunch of things about the experience Brain-1 was undergoing. These are all fuzzy, high-level properties, which admit of edge cases. But I'm not seeing what's gained by therefore concluding "I should anticipate every experience, even ones that have no causal connection to mine and no shared memories and no shared personality traits". Tables are a fuzzy and high-level concept, but that doesn't mean that every object in existence is a table. It doesn't even mean that every object is slightly table-ish. A photon isn't "slightly table-ish", it's just plain not a table. But they don't have the anticipation-related properties I listed above; so what hypotheses are we distinguishing by updating from "these experiences aren't mine" to "these experiences are mine"? Maybe the update that's happening is something like: "Previously it felt to me like other people's experiences weren't fully r
  I'll take a stab at explaining this with a simple thought experiment. Say there are two people, Alice and Bob, each with their own unique brain states. If Alice's brain state changes slightly, from getting older, learning something new, losing some neurons to a head injury, etc, she will still be Alice. Changing, adding, or removing a neuron does not change this fact. Now what if instead part of her brain state was changing slowly to match Bob's? You could think of this as incrementally removing Alice's neurons and replacing them with a copy of Bob's, I find it hard to believe that any discrete small change will make Alice's conscious experience suddenly disappear, and by the end of it she will have the exact same brain state as Bob. If you believe that when Bob steps into a teleporter that also makes a copy, they are both the same Bob, then it is reasonable to assume that this transformed Alice is also Bob. Then for the same reason your older self is the same "self" as your younger self, the younger Alice is also Bob. The transition between their brain states doesn't even need to happen, it just has to be possible. From here it is easy to extrapolate that all brain states are the same "self".
2Adele Lopez
I would say that Alice's conscious experience is unlikely to suddenly disappear under this transformation, and that it could even be done in a way so that their experience was continuous. However, Alice-memories would gradually fade out, Bob-memories would gradually fade in, and thought patterns would slowly shift from Alice-like to Bob-like. At the end, the person would just be Bob. Along the way, I would say that Alice gradually died (using an information-theoretic definition of death). The thing that is odd when imagining this is that Alice never experiences her consciousness fading. The main thing I think your thought experiment demonstrates is that our sense of self is not solely defined by continuity of consciousness.
I apologize for not getting back to you sooner, I didn’t notice your reply until yesterday. And I apologize for the length of my response, too - I bolded the most important parts. Re: Whether there is empirical difference between worlds where OI is true and where OI is false. The difference between all experiences being mine and only some being mine is that if all experiences are mine, then they all exist in the same way this experience now exists, i.e. for me (where me = just this immediacy/this-here-now character, i.e. the way it exists, NOT Edralis's memories, personality etc.). There is no empirical difference in the usual sense, since the way experiences exist cannot be objectively assessed. I can’t be sure that you even have any experiences – this is not something that is available for empirical investigation in the way I can assess e.g. the number of someone’s fingers. And I can’t know that, given there are experiences from that point of view, that they exist in the same way as this experience, does, i.e. for me.  That is only clear in those experiences. If I am there, I do ultimately know that I am there (obviously) – but I have no way to know that when experiencing this person, Edralis. So the empirical difference in the usual sense between OI being true and not being true is none. However, there are other than empirical facts. The existential difference between those two worlds is vast. If OI is true, then I (i.e. the thisness, the here-now-this that at least Edralis's experiences have) am Rob Bensinger, and everybody else – if it’s not true, then I am not. The difference is in the being of those experiences, in how they exist. But since experiences (consciousness) don’t exist empirically (or better: objectively), there is no empirical (objective) difference. There is existential, subjective difference, though. That is not what I mean when I think about anticipating a future brain state. What I am interested in is not the content of experience, but how t

So if something makes no physical difference to my current brain-state, and makes no difference to any of my past or future brain-states, then I think it's just crazy talk to think that this metaphysical bonus thingie-outside-my-brain is the crucial thing that determines whether I exist, or whether I'm alive or dead, etc.

There is one important aspect where it does make a difference. A difference in social reality. The brain states progress in a physically determined way. There is no way they could have progressed differently. When a "decision is made" by t... (read more)

Guys, social reality is one if not the cause of the self: Robin Hanson: PubMed: The essential moral self
This is a very interesting paper, thanks.

When faced with confusing conundrums like this, I find it useful to go back to basics: evolutionary psychology. You are a human, that is to say, you're an evolved intelligence, one evolved as a helpful planning-and-guidance system for a biological organism, specifically a primate. Your purpose, evolutionarily, is to maximize the evolutionary fitness of your genes, i.e. to try your best to pass them on successfully. You have whole bunch of drives/emotions/instincts that were evolved to, on the African Savannah, approximately maximize that fitness. Even in o... (read more)

I'm still struggling with this.  I'm fine with the notion that you could, in theory, teleport a copy of me across the universe and to that copy there would be a sense of continuity.  But your essay didn't convince me that the version of me entering the teleporter would feel that continuity.  To make it explicit, say you get into that teleporter and due to a software bug it doesn't "deconstruct" you up teleportation.  Here you are on this end and the technician says "trust me, you were teleported".  He then explains that due to inte... (read more)

3Seth Herd
The key to this koan (at least for me) is undoing the assumption that there can be only one of you. There's one of you that steps in and one that steps out. And they're the same you. What I value about me is the pattern of beliefs, memories, and values. The other me has has an identical brain state, so all of those. It is simply another me. I care about the second one pretty much exactly as much as I care about the same pattern continuing in a more similar location and with more similar molecules instantiating the pattern. That's because I care far less about where I am and which molecules I'm made of than the pattern of identity in my mind/brain. The same as you can have two of anything else that's close-enough-for-the-purpose. I can have two rocks if I don't care about the difference in their molecular makeup. I can have two mes. Yes, you have some sort of shared consciousness with the copy; it's the same shared consciousness between the you of today and the you that wakes up tomorrow. It doesn't imply sharing events that happen simultaneously or anything mystical about "sharing consciousness". That's why I'd happily step into the destructive teleporter if I was certain the copy on the other side would have exactly my mind-pattern, including memories, beliefs, and values. That's me.
These statements make the most sense only in the standard LW-computationalist frame, which reads to me as substantively anti-physicalist and mostly unreasonable to believe in, for reasons building off of what I sketched out in a comment to Ruby. But, in any case, I can concede it for now, if only for purposes of this conversation. The attempted mind-reading of others is (justifiably) seen as rude in conversations over the Internet, but I must nonetheless express very serious skepticism about this claim, as it's currently written. For one, I do not believe that "beliefs" and "values" ultimately make sense as distinct, coherent concepts that carve reality at the joints. This topic has been talked about before on LW a number of times, but I still fully endorse Charlie Steiner's distillation of it in his excellently-written Reducing Goodhart sequence: I expanded upon some of these ideas in a rather long comment I wrote to Wei Dai on the question of values and the orthogonality thesis: Moreover, you are explicitly claiming that your values are not indexical, which is rather unlikely in its own right, conflicts very strongly with my intuition (and, I would expect, with that of the vast majority of "regular", non-rationalist people), and certainly seems to disvalue (or even completely ignore) the relevance of continuous subjective experience. Put more clearly, if I were to be in such a spot, and one of my "copies" were told to choose between being tortured or having the other copy be tortured instead, it would certainly choose the latter option, and I suspect this to be the case for ~ every other person as well (with apologies for a slight generalization from one example). In any case, the rather abstract "beliefs, memories and values" you solely purport to value fit the category of professed ego-syntonic morals much more so than the category of what actually motivates and generates human behavior, as Steven Byrnes explained in an expectedly outstanding way: Steve al
3Seth Herd
I don't have time to respond to all of this. I don't disagree with any particular claim you've made there. I value the continuity of experience as much as you; the experience of a pattern continuing down to the most minute detail is more continuous than when we fall asleep, have some half-conscious and fully unconscious states, and wake up as an approximate but less precise continuation of the mental pattern we were when we went to sleep. The fine distinctions in beliefs and values don't matter. I agree with all of your statements about the vagaries and confusions about beliefs and values, but they're not relevant here. That perfectly duplicated pattern carries all of them, stated and unstated, complex and simple. Every memory. There's nothing else to value, except for continuity in space and time. I'd rather be me waking up in Des Moines in a month than stay where I am and get brain damage (and loss of self) in one minute.. I confess that I don't love going to sleep, but I assume that you also don't consider it similar to death. You've got a lot of questions to raise, but no apparent alternative. Your mind is a pattern. That pattern is instantiated in matter. Reproduce the matter, you've reproduced the mind. That's not anti-physicalist, it's just how physics of information processing works. The only alternative is positing a mind-pattern that's not tightly connected to matter - but that helps explain nothing. The physical world works just fine for instantiating the information processing you need to create a mind that is self-aware and simulates its environment like humans seem to do. I don't disagree with anything you've said; it's just not an alternative view. You're fighting against the counterintuitive conclusion. Sure I'd rather have a different version of me be tortured; it's slightly different. But I won't be happy about it. And my intuition is still drawn toward continuity being important, even though my whole rational mind disagrees. I've been back and
Non computationalism physicalism is an alternative to either or both the computationalist theories. (That performing a certain class of computations is sufficient to be conscious in general, or that performing a specific one is sufficient to be a particular conscious individual. Computation as a theory of consciousness qua awareness isn't known to be true, and even if it is assumed, it doesn't directly give you a theory of personal identity). The non existence, or incoherence, of personal identity is another. There doesn't have to be an answer to "when is a mind me". Note that no one except andeslodes is arguing against copying. The issue is when a mind is me, the person typing this, not a copy-of-me. Well, that's only copying. Consciousness, qua Awareness, and Personal Identity are easily confused, not least because both are often called "consciousness". A computational theory of consciousness is sometimes called on to solve the second problem, the problem of personal identity. But there is no strong reason to think a computational duplicate of you, actually is you, since there is no strong reason to think any other kind of duplicate is. Qualitative identity is a relationship between two or more things that are identical in all their properties. Numerical identity is the relationship a thing has only to itself. The Olsen twins enjoy qualitative identity; Stephanie Germanota and Lady Gaga have numerical identity. The trick is to jump from qualitative identity to numerical identity, because the claim is that a computational duplicate of you, is you, the very same person. Suppose you found out you had an identical twin. You would not consider them to be you yourself. Likewise for a biological clone. A computational duplicate would be lower resolution still, so why would it be you? The major problem is that you and your duplicate exist simultaneously in different places, which goes against the intuition that you are a unique individual. You don't really believe
2Seth Herd
If you're not arguing against a perfect copy being you, then I don't understand your position, so much of what follows will probably miss the mark. I had written more but have to cut myself off since this discussion is taking time without having much odds of improving anyone's epistemics noticably. The Olson twins are do not at all have qualitative identity. They have different minds: sets of memories, beliefs, and values. So I just don't know what your position is. You claim that there doesn't need to be an answer; that seems false, as you could have to make decisions informed by your belief. You currently value your future self more than other people, so you act like you believe that's you in a functional sense. Are you the same person tomorrow? It's not an identical pattern, but a continuation. I'm saying it's pretty-much you because the elements you wouldn't want changed about yourself are there. If you value your body or your continuity over the continuity of your memories, beliefs, values, and the rest of your mind that's fine, but the vast majority will disagree with you on consideration. Those things are what we mean by "me". I certainly do believe in the plural I (under the special cirrumstance I discussed); we must be understanding something differently in the torture question. I don't have a preference pre-copy for who gets tortured; both identical future copies are me from my perspective before copying. Maybe you're agreeing with that? After copying, we're immediately starting to diverge into two variants of me, and future experiences will not be shared between them. I was addressing a perfect computational copy. An imperfect but good computational copy is higher resolution, not lower, than a biological twin. It is orders of magnitude more similar to the pattern that makes your mind, even though it is less similar to the pattern that makes your body. What is writing your words is your mind, not your body, so when it says "I" it meets the mind. Non
I agree that this conversation, as currently started, is unlikely to lead to anything more productive. As such, I'll keep my response here brief [1], in case you want to use it as a starting point if you ever intend for us to talk about it again. As I read these statements, they fail to contend with a rather basic map-territory distinction that lies at the core of "physics" and "computation."  The basic concept of computation at issue here is a feature of the map you could use to approximate reality (i.e., the territory) . It is merely part of a mathematical model that, as I've described in response to Ruby earlier, represents a very lossy compression of the underlying physical substrate [2]. This is because, in this restricted and epistemically hobbled ontology, what is given inordinate attention is the abstract classical computation performed by a particular subset of the brain's electronic circuit. This is what makes it anti-physicalist, as I have explained: To make it even more explicit, this interpretation of the computationalist perspective (that the quantum stuff doesn't matter etc) was confirmed as accurate by its proponents. So when you talk about a "pattern instantiated by physics as a pure result of how physics works", you're not pointing to anything meaningful in the territory, rather only something that makes sense in the particular ontology you have chosen to use to view it through, a frame that I have explained my skepticism of already. 1. ^ This will be my final comment in this thread, regardless of what happens. 2. ^ Put differently, "computation" is not an ontologically primitive concept in reality-as-it-is, but only in mathematical approximations of it that make specific assumptions about what does and doesn't exist. Those assumptions can be sometimes justified in terms of intuitive appeal, expediency of calculation etc, but reifying them as unchallengeable axioms of the universe rather than of your model of it is wrong.
I disagree that your mind is "a pattern instantiated in matter." Your mind is the matter. It's precisely the assumption that the mind is separable from the matter that I would characterize as non-physicalist.
2Seth Herd
Terminology aside, I think if you examine this carefully it's incoherent. Do you think a successful upload would say things like "I'm still me!" and think thoughts like "I'm so glad I payed extra to give myself cool virtual environment options"? That seems like an inevitability if the causal patterns of your mind were captured. And it would be tough to disagree with a thing claiming up and down it's you, citing your most personal memories as evidence.
A successful upload (assuming this is physically possible, which is not a settled question) would remember my same memories and have my same personality traits; however, that would not mean my mind had been unwound from the matter and transferred to it, but rather that my mind had been duplicated in silico.
2Seth Herd
Yes, it's a duplicate which will also be you from your current perspective. If you duplicated your car tomorrow you'd have two cars; if you duplicate your mind tomorrow you need to plan on there being two yous tomorrow.
No; it will remember my life but I will not go on to experience its experiences. (Similarly, if I duplicate my car and then destroy the original, its engine does not continue on to fire in the duplicate; the duplicate has an engine of its own, which may be physically identical but is certainly not the same object).
2Seth Herd
Okay, so would you say that the you of today goes on to experience the you-of-tomorrow's experiences? I think the relationship is the same to a perfect duplicate. The duplicate is no less you than the you of tomorrow is. They are separate people from their perspective after duplication, but almost-the-same-person to a much greater degree than twins. You (pre-duplication) will go on to have two separate sets of experiences. Both are you from your current perspective before duplication; you should give them equal consideration in your decisions, since the causal relationship is the same in both ways between you and the duplicate as to your self of tomorrow. Consider the case where the duplicate is teleported to your location and vice versa during duplication. Then just location swapped while you're asleep. And consider that you wouldn't care a whit if every molecule of your body was Theseus-swapped one by one for identical molecules in identical locations and roles while you slept.
No; I, pre-duplication, exist in a single body, and will not post-duplication have my consciousness transferred over to run in two. There will just be an identical copy. If the original body dies, one of me will also die. The causal relationship between me and myself tomorrow is not the same is the causal relationship between me and my duplicate tomorrow, because one of those is a physical object which has continuity over time and one of those is a similar physical object which was instantiated de novo in a different location when the teleporter was run. The mind is not a program which runs on the meat computer of the brain and could in principle be transferred to a thumb drive if we worked out the format conversions; the mind is the meat of the brain.
Realistically I doubt you'd even need to be sure it works, just reasonably confident.  Folks step on planes all the time and those do on rare occasion fail to deliver them intact at the other terminal.
Within this framework, whether or not you "feel that continuity" would mostly be a fact about the ontology your mindstate uses thinking about teleportation.  Everything in this post could be accurate and none of it would be incompatible with you having an existential crisis upon being teleported, freaking out upon meeting yourself, etc. Nor does anything here seem to make a value judgement about what the copy of you should do if told they're not allowed to exist.  Attempting revolution seems like a perfectly valid response; self defense is held as a fairly basic human right after all. (I'm shocked that isn't already the plot of a sci-fi story.) It would also be entirely possible for both of your copies to hold conviction that they're the one true you - Their experiences from where they sit being entirely compatible with that belief. (Definitely the plot of at least one Star Trek episode.) There's not really any pressure currently to have thinking about mind copying that's consistent with every piece of technology that could ever conceivably be built.  There's nothing that forces minds to have accurate beliefs about anything that won't kill them or wouldn't have killed their ancestors in fairly short order.  Which is to say mostly that we shouldn't expect to get accurate beliefs about weird hypotheticals often without having changed our minds at least once.

"You should anticipate having both experiences" sounds sort of paradoxical or magical, but I think this stems from a verbal confusion.

You can easily clear this confusion if you rephrase it as "You should anticipate having any of these experiences". Then it's immediately clear that we are talking about two separate screens. And it's also clear that our curriocity isn't actually satisfied. That the question "which one of these two will actually be the case" is still very much on the table.

Rob-y feels exactly as though he was just Rob-x, and Rob-z also feels

... (read more)
3Rob Bensinger
This introduces some other ambiguities. E.g., "you should anticipate having any of these experiences" may make it sound like you have a choice as to which experience to rationally expect. ... And the answer is "both of these will actually be the case (but not in a split-screen sort of way)". Your rephrase hasn't shown that there was a question left unanswered in the original post; it's just shown that there isn't a super short way to crisply express what happens in English, you do actually have to add the clarification. Yep, I think this is a perfectly fine way to think about the thing.

abstract redescriptions of ordinary life

See Reality is Normal 

If a brain-state A has quasi-sensory access to the experience of another brain-state B — if A feels like it "remembers" being in state B a fraction of a second ago — then A will typically feel as though it used to be B.

This suggests a way to add a perception of "me" to LLMs, robots, etc., by providing a way to observe the past states in sufficient detail. Current LLMs have to compress this into the current token, which may not be enough. But there are recent extensions that seem to do something like continuous short-term memory, see e.g., Leave No Context Behind - A Comment.

I am currently working on a similar post that comes from an eliminative perspective.

There are other reasons to be wary of consciousness and identity-altering stuff. 

I think under a physical/computational theory of consciousness, (ie. there's no soul or qualia that have provable physical effects from the perspective of another observer) the problem might be better thought of as a question of value/policy rather than a question of fact. If teleportation or anything else really affects qualia or any other kind of subjective awareness that is not purely dependent on observable physical facts, whatever you call it, you wouldn't be able to... (read more)

I claim you are in fact highly confused about what a self is, in a way that makes an almost-correct reasoning process produce nonsense outcomes because of an invalid grounding in the transition processes underneath the mind which does not preserve truth values regarding amounts of realityfluid.

update 7d after writing this comment in my comment below. strikethrough added to this comment where I've changed my mind.

If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biologic

... (read more)
2the gears to ascension
Update: a friend convinced me that I really should separate my intuitions about locating patterns that are exactly myself from my intuitions about the moral value of ensuring I don't contribute to a decrease in realityfluid of the mindlike experiences I morally value, in which case the reason that I selfishly value causal history is actually that it's an overwhelmingly predictive proxy for where my self-pattern gets instantiated, and my moral values - an overwhelmingly larger portion of what I care about - care immensely about avoiding waste, because it appears to me to be by far the largest impact any agent can have on what the future is made of. Also, I now think that eating is a form of incremental uploading.

If we were just talking about word definitions and nothing else, then sure, define “self” however you want. You have the universe’s permission to define yourself into dying as often or as rarely as you’d like, if word definitions alone are what concerns you.

But this post hasn’t been talking about word definitions. It’s been talking about substantive predictive questions like “What’s the very next thing I’m going to see? The other side of the teleporter? Or nothing at all?”

There should be an actual answer to this, at least to the same degree there’s an ans

... (read more)
2Rob Bensinger
Which things count as "I" isn't an arbitrary definition; it's just a fuzzy natural-language concept. (I guess you can call that "arbitrary" if you want, but then all the other words in the sentence, like "doorway" and "step", are also "arbitrary".) Analogy: When you're writing in your personal diary, you're free to define "table" however you want. But in ordinary English-language discourse, if you call all penguins "tables" you'll just be wrong. And this fact isn't changed at all by the fact that "table" lacks a perfectly formal physics-level definition. The same holds for "Will Rob Bensinger's next experience be of sitting in his bedroom writing a LessWrong comment, or will it be of him grabbing some tomatoes in a supermarket in Beijing?" Terms like 'Rob Bensinger' and 'I' aren't perfectly physically crisp — there may be cases where the answer is "ehh, maybe?" rather than a clear yes or no. And if we live in a Big Universe and we allow that there can be many Beijings out there in space, then we'll have to give a more nuanced quantitative answer, like "a lot more of Rob's immediate futures are in his bedroom than in Beijing". But if we restrict our attention to this Beijing, then all that complexity goes away and we can pretty much rule out that anyone in Beijing will happen to momentarily exhibit exactly the right brain state to look like "Rob Bensinger plus one time step". The nuances and wrinkles don't bleed over and make it a totally meaningless or arbitrary question; and indeed, if I thought I were likely to spontaneously teleport to Beijing in the next minute, I'd rightly be making very different life-choices! "Will I experience myself spontaneously teleporting to Beijing in the next second?" is a substantive (and easy) question, not a deep philosophical riddle. Not all possible experiences; just all experiences of brains that have the same kinds of structural similarities to your current brain as, e.g., "me after I step through a doorway" has to "me be
You're also free to define "I" however you want in your values. You're only wrong if your definitions imply wrong physical reality. But defining "I" and "experiences" in such a way that you will not experience anything after teleportation is possible without implying anything physically wrong. You can be wrong about physical reality of teleportation. But even after you figured out that there is no additional physical process going on that kills your soul, except for the change of location, you still can move from "my soul crashes against an asteroid" to "soul-death in my values means sudden change in location" instead of to "my soul remains alive". It's not like I even expect you specifically to mean "don't liking teleportation is necessary irrational" much. It's just that saying that there should be an actual answer to questions about "I" and "experiences" makes people moral-realist.
5Rob Bensinger
Sort of! * It's true that no law of nature will stop you from using "I" in a nonstandard way; your head will not explode if you redefine "table" to mean "penguin". * And it's true that there are possible minds in abstract mindspace that have all sorts of values, including strict preferences about whether they want their brain to be made of silicon vs. carbon. * But it's not true that humans alive today have full and complete control over their own preferences. * And it's not true that humans can never be mistaken in their beliefs about their own preferences. In the case of teleportation, I think teleportation-phobic people are mostly making an implicit error of the form "mistakenly modeling situations as though you are a Cartesian Ghost who is observing experiences from outside the universe", not making a mistake about what their preferences are per se. (Though once you realize that you're not a Cartesian Ghost, that will have some implications for what experiences you expect to see next in some cases, and implications for what physical world-states you prefer relative to other world-states.)
Why not both? I can imagine that someone would be persuaded to accept teleportation/uploading if they stopped believing in physical Cartesian Ghost. But it's possible that if you remind them that continuity of experience, like table, is just a description of physical situation and not divinely blessed necessary value, that would be enough to tip the balance toward them valuing carbon or whatever. It's bad to be wrong about Cartesian Ghosts, but it's also bad to think that you don't have a choice about how you value experience.
The problem was that you first seemed to belittle questions about word meanings ("self") as being "just" about "definitions" that are "purely verbal". Luckily now you concede that the question about the meaning of "I" isn't just about (arbitrary) "definitions", which makes calling it a "purely verbal" (read: arbitrary) question inappropriate. Now of course the meaning of "self" is no more arbitrary than the meaning of "I", indeed those terms are clearly meant to refer to the same thing (like "me" or "myself"). The wider point is that the following seems not true: Wenn we evaluate statements or questions of any kind, including the one above, we need to know two things: 1) Its meaning, in particular the meaning of the involved terms, 2) what the empirical facts are. But we already know all the empirical facts: Someone goes into the teleporter, a bit later someone comes out at the other end and sees something. So the issue can only be about the semantic interpretation of that question, about what we mean with expressions like "I will see x". Do we mean "A future person that is psychologically continuous with current-me sees x"? That's not an empirical question, it's a semantic one, but it's not in any way arbitrary, as expressions like "just about definitions" or "purely verbal" would suggest. Conceptual analysis is neither arbitrary nor trivial.
7Rob Bensinger
I did no such thing! Read the blog post at the top of this page! It's my attempt to answer the question of when a mind is "me", and you'll notice it's not talking about definitions. Nope! There are two perspectives here: 1. "I don't want to upload myself, because I wouldn't get to experience that uploads' experiences. When I die, this stream of consciousness will end, rather than continuing in another body. Physically dying and then being being copied elsewhere is not phenomenologically indistinguishable from stepping through a doorway." 2. "I do want to upload myself, because I would get to experience that uploads' experiences. Physically dying and then being copied myself is phenomenologically indistinguishable from stepping through a doorway." The disagreement between these two perspectives isn't about word definitions at all; a fear that "when my body dies, there will be nothing but oblivion" is a very real fear about anticipated experiences (and anticipated absences of experience), not a verbal quibble about how we ought to define a specific word. But it's also a bit confusing to call the disagreement between these two perspectives "empirical", because "empirical" here is conflating "third-person empirical" with "first-person empirical". The disagreement here is about whether a stream of consciousness can "continue" across temporal and spatial gaps, in the same way that it continues when there are no obvious gaps. It's about whether there's a subjective, experiential difference between stepping through a doorway and using a teleporter. The thing I'm arguing in the OP is that there can't be an experiential difference here, because there's no physical difference that could be underlying the supposed experiential difference. So the disagreement about the first-person facts, I claim, stems from a cognitive error, which I characterize as "making predictions as though you believed yourself to be a Cartesian Ghost (even if you don't on-reflection endorse the
Is there even anybody claiming there is an experiential difference? It seems you may attacking a strawman. The alternative to this is that there is a disagreement about the appropriate semantic interpretation/analysis of the question. E.g. about what we mean when we say "I will (not) experience such and such". That seems more charitable than hypothesizing beliefs in "ghosts" or "magic".
7Rob Bensinger
Yep! Ask someone with this view whether the current stream of consciousness continues from their pre-uploaded self to their post-uploaded self, like it continues when they pass through a doorway. The typical claim is some version of "this stream of consciousness will end, what comes next is only oblivion", not "oh sure, the stream of consciousness is going to continue in the same way it always does, but I prefer not to use the English word 'me' to refer to the later parts of that stream of consciousness". This is why the disagreement here has policy implications: people with different views of personal identity have different beliefs about the desirability of mind uploading. They aren't just disagreeing about how to use words, and if they were, you'd be forced into the equally "uncharitable" perspective that someone here is very confused about how relevant word choice is to the desirability of uploading. I didn't say that the relevant people endorse a belief in ghosts or magic. (Some may do so, but many explicitly don't!) It's a bit darkly funny that you've reached for a clearly false and super-uncharitable interpretation of what I said, in the same sentence you're chastising me for being uncharitable! But also, "charity" is a bad approach to trying to understand other people, and bad epistemology can get in the way of a lot of stuff.
5Rob Bensinger
As a test, I asked a non-philosopher friend of mine what their view is. Here's a transcript of our short conversation: https://docs.google.com/document/d/1s1HOhrWrcYQ5S187vmpfzZcBfolYFIbeTYgqeebNIA0/edit  I was a bit annoyingly repetitive with trying to confirm and re-confirm what their view is, but I think it's clear from the exchange that my interpretation is correct at least for this person.
This doesn't show they believe there is a difference in experience. It can be simply a different analysis of the meaning of "the current stream of consciousness continuing". That's a semantic difference, not an empirical one.

If we will very quickly constantly replace a mind with its copies, the mind may not have subjective experiences. Why I think that?

Subjective experience appear only when a mind moves from the state A.1 to the state A.2. That is, between A.I (I see an apple) electric signals move through circuits and in the A.2 moment I say "I see an apple!" Subjective experience of the color of apple is happening after A.1 but before A.2. 

Frozen mind in A.1 will not have subjective experience.

Now if I replace this process with a series of snapshots of the brain-states,... (read more)

I think I basically agree with everything here, but probably less confidently for you, such that I would have a pretty large bias against destructive whole brain emulation, with the biggest crux being how anthropics works over computations.

You say that there’s no XML tag specifying whether some object is “really me” or not, but a lighter version of that—a numerical amplitude tag specifying how “real” a computation is—is the best interpretation we have for how quantum mechanics works. Even though all parts of me in the wavefunction are continuations of the ... (read more)

I am surprised I didn't find any reference to Tim Urban's "Wait But Why" post What Makes You You.

In short, he argues that "you" is your sense of continuity, rather than your physical substance. He also argues that if (somehow) your mind was copied&pasted somewhere else, then a brand new "not-you" would be born - even though it may share 100% of your memory and behaviour.
In that sense, Tim argues that Theseus' ship is always "one" despite all its parts are changed over time. If you were to disassemble and reassemble the ship, it would lose its continuity and it could arguably be considered a different ship.

You seem to make a strong assumption that consciousness emerges from matter. This is uncertain. The mind body problem is not solved.

Nor is there a law of physics saying "your subjective point of view immediately blips out of existence and is replaced by Someone Else's point of view if your spacetime coordinates change a lot in a short period of time (even though they don't blip out of existence when your spacetime coordinates change a little or change over a longer period of time)".

I feel like this isn't a fair comparison, as if I were cloned completely and relocated (teleportation), I wouldn't expect to experience both original me and cloned me.
The best analogy I can think of is as fo... (read more)

a magical Cartesian ghost

for people who haven't made the intuitive jump that you seem to try to convey, this may seem a somewhat negative expression, which could lead to aversion. I recommend another expression such as "the Cartesian homunculus."  

in short, it seems to me that the crux of the argument comes down to whether there is physiological continuity of self or 'consciousness' for lack of a better word.  

I suspect this will also actually have very relevant applications in field such as cryonics which adds an additional layer of complexity because all metabolic processes will completely cease to function.
Conducting the duplication experiment during sleep (or any altered state of consciousness) is interesting but nevertheless there is clearly physical (physiological) continuity of the... (read more)

Another consideration, though maybe not a fundamental one, is that past and future selves are the only beings we know for sure that we have lots of subjunctive dependence with, just from "structural" similarity like calculators from the same factory (to use an example from the TDT paper). Tumblr usr somnilogical elaborated on this a bit, concluding "future selves do help past selves!" An upload is a future self in the way that matters for this conclusion.

Your topline answers to the questions you assume xlr8harder cares about more seem similar to Holden Karnofsky's, and I haven't seen his essay on this mentioned so in this thread so I thought it'd be useful to link it here: What counts as death? An unconventional but simple take on personal identity, that dissolves most paradoxes

My philosophy on "what counts as death" is simple, though unconventional, and it seems to resolve most otherwise mind-bending paradoxical thought experiments about personal identity. It is the same basic idea as the one advanced by

... (read more)

Intriguing post, but we should approach these topics with extreme epistemic humility. Our understanding is likely far more limited and confused than we realize:

1. Abstractions vs. reality: Concepts like "self" and "consciousness" are abstractions, not reality. As Kosoy analogizes, these might be like desktop icons - a user interface bearing little resemblance to underlying hardware.

2. Mathematical relations: Notions of "copy" may be a confused way to discuss identity. "Consciousness" could be a mathematical relation where only identities exist, with "copie... (read more)

It's only if I'm in my brain, just an ordinary part of physics, that mind uploading makes sense as a way to extend my lifespan.

It's precisely if you're in your brain that mind uploading doesn't make sense; if you are your brain, the destruction of your brain will also kill you.

Taking a step back, we can ask: what physical mechanism makes it feel as though I'm persisting over time?

The actual physical organism which you are persists over time, and you are not a separate thing from the physical organism. This does not apply in the teleporter case, because in the teleporter case the relevant physical organism is disassembled atom-by-atom and a duplicate is assembled elsewhere. (The duplicate is, well, a duplicate; if it's assembled on the other side without disassembling the initial person, there will then be two people. They ... (read more)

The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

I don’t see why split-screen mode is crazy talk at all. Is it just because it would imply faster-than-light communication? With our understanding of physics incomplete, I remain agnostic on the existence of FTL, so I wouldn’t rule this out. But even more than that, I’d propose that if there is one observer, there does not even need to be FTL communication in the first place, because it is just that the observer is in more than one place at once, similarly to how a wormhole does not necessitate true FTL. What are the other objections?

The belief system which... (read more)

It's not just that it implies faster-than-light communication, it's that it implies communication at all. Experiencing both bodies at the same time, you will be able to take actions in one body that you wouldn't have done without the other one. It seems odd that with no biological changes to your brain, the mere existence of another similar brain changes how this one functions. Why would they be linked? This implies the observer is some external soul-like thing that can manipulate matter. If you can't take actions based on your conscious experience, it implies the observer is dissociated from the brain and not created from it or able to interact with it. I can definitely imagine a world where this is true, but it seems extremely unlikely based on what we currently know.
1Flying Pen and Paper
Yes, it would imply the observer is external, but then it also would not change anything about how the brain functions. (Or vice versa, but I prefer this one.) I am unconvinced of the truth of what you say in the last sentence of your second paragraph. Either way, whether or not it might seem implausible, my question is why it is, or is not, implausible. Why exactly, based on what we currently know, is this extremely unlikely?

Does Scenario 2 imply some kind of spooky action at a distance? How is information from Rob-z transmitted to the homonculus over large distances? Are there 2 homoncului now that communicate what they see to each other?

Doesn’t scenario 2 imply Rob-x has actually functionally died? Which would make this the scenario where you don’t care about what happens to Rob-z and y because Rob-x now experiences oblivion?

https://existentialcomics.com/comic/1 Related comic exploring this idea

What concrete fact about the physical world do you think you're missing? What are you ignorant of?

Let's flip very unfair quantum coin with 1:2^1000000 heads to tails chances (that would require quite an engineering feat to prepare such a quantum state, but it's theoretically possible). You shouldn't expect to see heads if the quantum state is prepared correctly, but the post-flip universe (in MWI) contains a branch where you see heads. So, by your logic, you should expect to see both heads and tails even if the state is prepared correctly.

What I do not kno... (read more)

Consider the teleporter as a machine that does two things: deconstructs an input i and constructs an output o. 
If you divide the machine logically into these two functions, d and c, which are responsible for deconstructing and constructing respectively, you have four ways the machine could function or not function:

If neither d or c work, the machine doesn't do anything. 

If d works but c doesn't, the machine definitely kills or destroys the input person. 

If d doesn't work and c does, the machine makes a copy of the person. If a being walked i... (read more)

But I hope the arguments I've laid out above make it clear what the right answer has to be: You should anticipate having both experiences.

Some quantum experiments allow us to mostly anticipate some outcomes and not others. Either quantum physics doesn’t work the way Eliezer thinks it works and the universe is very small to not contain many spontaneously appearing copies of your brain, or we should be pretty surprised to continually find ourselves in such an ordered universe, where we don’t start seeing white noise over and over again.

I agree that if the... (read more)

2Vanessa Kosoy
The solution is here. In a nutshell, naive MWI is wrong, not all Everett branches coexist, but a lot of Everett branches do coexist s.t. with high probability all of them display expected frequencies.
1Mikhail Samin
I can imagine this being the solution, but * this would require a pretty small universe * if this is not the solution, my understanding is that IBP agents wouldn’t know or care, as regardless of how likely it is that we live in naive MWI or Tegmark IV, they focus on the minimal worlds required. Sure, in these worlds, not all Everett branches coexist, and it is coherent for an agent to focus only on these worlds; but it doesn’t tell us much about how likely we’re in a small world. (I.e., if we thought atoms are ontologically basic, we could build a coherent ASI that only cared about worlds with ontologically basic atoms and only cared about things made of ontologically basic atoms. After observing the world, it would assume it’s running in a simulation of a quantum world on a computer build of ontologically basic atoms, and it would try to influence the atoms outside the simulation and wouldn’t care about our universe. Some coherent ASIs being able to think atoms are ontologically basic shouldn’t tell us anything about whether atoms are indeed ontologically basic.) Conditional on a small universe, I would prefer the IBP explanation (or other versions of not running all of the branches and producing the Born rule). Without it, there’s clearly some sort of sampling going on.
2Vanessa Kosoy
Not sure what you mean by "this would require a pretty small universe". If we live in naive MWI, an IBP agent would not care for good reasons, because naive MWI is a "library of babel" where essentially every conceivable thing happens no matter what you do. Also not sure what you mean by "some sort of sampling". AFAICT, quantum IBP is the closest thing to a coherent answer that we have, by a significant margin.
Isn't the frequency of amplitude-patterns changes depending on what you do? So an agent can care about that instead of point-states.
1Mikhail Samin
I mean if the universe is big enough for every conceivable thing to happen, then we should notice that we find ourselves in a surprisingly structured environment and need to assume some sort of an effect where if a cognitive architecture opens its eyes, it opens its eyes in a different places with the likelihood corresponding to how common these places are (e.g., among all Turing machines). I.e., if your brain is uploaded, and you see a door in front of you, and when you open it, 10 identical computers start running a copy of you each: 9 show you a green room, 1 shows you a red room, you expect that if you enter a room and open your eyes, in 9/10 cases you’ll find yourself in a green room. So if it is the situation we’re in- everything happens- then I think a more natural way to rescue our values would be to care about what cognitive algorithms usually experience, when they open their eyes/other senses. Do they suffer or do they find all sorts of meaningful beauty in their experiences? I don’t think we should stop caring about suffering just because it happens anyway, if we can still have an impact on how common it is. If we live in a naive MWI, an IBP agent doesn’t care for good reasons internal to it (somewhat similar to how if we’re in our world, an agent that cares only about ontologically basic atoms doesn’t care about our world, for good reasons internal to it), but I think conditional on a naive MWI, humanity’s CEV is different from what IBP agents can natively care about.
3Vanessa Kosoy
Your reasoning is invalid, because in order to talk about updating your beliefs in this context, you need a metaphysical framework which knows how to deal with anthropic probabilities (e.g. it should be able to answer puzzles in the vein of the anthropic trilemma according to some coherent, well-defined mathematical rules). IBP is such a framework, but you haven't proposed any alternative, not to mention an argument for why that alternative is superior.
I always thought that in naive MWI what matters is not whether something happens in absolute sense, but what Born measure is concentrated on branches that contain good things instead of bad things.
3Vanessa Kosoy
The problem is this requires introducing a special decision-theory postulate that you're supposed to care about the Born measure for some reason, even though Born measure doesn't correspond to ordinary probability.
Huh? The whole point of the Born rule is to get a set of ordinary probabilities, which you can then test frequentistically, over a run of experiments. Quantum mechanical measure-- amplitude-- isn't ordinary probability, but that's the thing you put into the Born rule, not the thing you get out of it. And it has it's own role, which is explaining how much contribution to a coherent superposition each component state makes. ETA There is a further problem interpreting the probabilities of fully decohered branches. (Calling then Everett branches is very misleading -- a clear theory of decoherence is precisely what's lacking in Everett's work) Whether you are supposed to care about them ethically is very unclear, since it is not clear how utilitarian style ethics would apply, even if you could make sense of the probabilities. But you are not supposed to care about them for the purposes of doing science, since they can no longer make any difference to your branch. MWI works like a collapse theory in practice. It's tempting to ethically discount low measure decoherent branches in some way, because that most closely approximates conventional single world utilitarianism -- that is something "naive MWI" might mean. However, one should not jump to the conclusion that something is true just because it is convenient. And of course, MWI is a scientific theory so it doesn't comes with built in ethics. The alternative view starts with the question of whether a person low measure world still count as a full.person? If they should not, is that because they are a near-zombie, with a faint consciousness that weighs little in a hedonic utilitarian calculus? If they are not such zombies, why would they not count as a full person -- the standard utilitarian argument that people in far-off lands are still moral patients seems to apply. Of course, MWI doesn't directly answer the question about consciousness. If "naive MWI" means the idea that any elementary interaction produces decohe
2Vanessa Kosoy
The topic of this thread is: In naive MWI, it is postulated that all Everett branches coexist. (For example, if I toss a quantum fair coin n times, there will be 2n branches with all possible outcomes.) Under this assumption, it's not clear in what sense the Born rule is true. (What is the meaning of the probability measure over the branches if all branches coexist?)

But it could matter if its digital vs continuous.  <OK longer post and some thoughts a bit off topic perhaps>

Your A,B,C,D ... leads to some questions about what is conscious (C) and what isn't. 

Where exactly does the system stop being conscious

1. Biological mind with neurons

2. Very high fidelity render in silicon with neurons modelled down to chemistry rather than just firing pulses

3. Classic neural net spiking approx done in discrete maths that appears almost indistinguishable to 1,2. Producing system states A,B,C,D

4. same as (3) but state... (read more)


Curated. I like "think clearly about confusing philosophical" topics and this post is a very well-written explainer. I think it's likely both correct and ought to be convincing. At the same time, I think it's somewhat incomplete and that a fuller version would make the case for why we should wholly believe that experience is determined by local brain state rather than treat it as an assumption. Beyond that, I think that the most convincing explainer on this topic would build on a fully satisfactory theory of consciousness that totally answers the hard problem and makes none of it feel mysterious at all. Still, glad to see this on LessWrong, taking us kind of bag to our roots.

I find this rather difficult to believe in light of andesoldes's excellent distillation of Rob's position and subsequent detailed and concrete explanation of why it seems wrong to have this degree of confidence in his beliefs. As TAG has written a number of times, the computationalist thesis seems not to have been convincingly (or even concretely) argued for in any LessWrong post or sequence (including Eliezer's Sequences). What has been argued for, over and over again, is physicalism, and then more and more rejections of dualist conceptions of souls.  That's perfectly fine, but "souls don't exist and thus consciousness and identity must function on top of a physical substrate" is very different from "the identity of a being is given by the abstract classical computation performed by a particular (and reified) subset of the brain's electronic circuit," and the latter has never been given compelling explanations or evidence. [1] This is despite the fact that the particular conclusions that have become part of the ethos of LW about stuff like brain emulation, cryonics etc are necessarily reliant on the latter, not the former.  As a general matter, accepting physicalism as correct would naturally lead one to the conclusion that what runs on top of the physical substrate works on the basis of... what is physically there (which, to the best of our current understanding, can be represented through Quantum Mechanical probability amplitudes), not what conclusions you draw from a mathematical model that abstracts away quantum randomness in favor of a classical picture, the entire brain structure in favor of (a slightly augmented version of) its connectome, and the entire chemical make-up of it in favor of its electrical connections. As I have mentioned, that is a mere model that represents a very lossy compression of what is going on; it is not the same as the real thing, and conflating the two is an error that has been going on here for far too long. Of course, it very w
I differ from Rob in that I do think his piece should have flagged the assumption of ~computationalism, but think the assumption is reasonable enough to not have argued for in this piece. I do think it is interesting philosophical discussion to hash it out, for the sake of rigor and really pushing for clarity. I'm sad that I don't think I could dive in deep on the topic right now. To answer your question in your other comment. I reckon with some time I could write an explainer for why we should very reasonable assume consciousness is the result of local brain stuff and nothing else (and also not quantum stuff), though I'd be surprised if I could easily write something so rigorous that you'd find it fully satisfactory.
Relatedly to my other comment, I'm curious if you (Ruby) think you would be capable of writing such a version. I'm obviously not asking you to actually write it (you probably have much better things to do with your time), but I do wonder what the answer to this question is, and if it is "no", then I would also want to ask whether you nonetheless think you have good reasons to believe that Rob's conclusion is correct (in light of the counterarguments and reasons for skepticism that have been brought up, and which seem to me like they would necessitate that exact "fuller version" to resolve).