[Epistemic Status: rambly and exploratory, hoping for some serious discussion]
It's 2030, the AGI takeoff ended badly and humanity is wiped out by a misaligned AGI. You die. The end.
The end? The above is a cliché many of us have in mind when discussing scenarios where death is involved. Death is often talked about as the final chapter. You're just... gone.
But what does death actually feel like from a subjective point of view?
In an infinite multiverse where every possible outcome is realized (e.g. a Tegmark level III/IV world), there are guaranteed to be instances where "you" are "resurrected", in the sense that there exist computations that possess your exact memory and personality before you died and that will continue to be run after "experiencing death". An example would be a resurrection simulation of you being carried out by an FAI in branches where humanity has succeeded in tackling the alignment problem. The probability of being resurrected does not matter, as long as it is non-zero you should expect to feel resurrected in an infinite multiverse since it is guaranteed to happen.
Therefore, we should expect "death" to feel like falling asleep, since we are guaranteed to continue experiencing observer moments after our death in our subjective point of view. If you allow identity to be extended over time (like almost all of us do) as a chain of similar observer-moments, then the logical conclusion is that you will never "die" in the sense of experiencing nothing. "Experiencing death" is a tautologically meaningless statement. Oblivion doesn't "feel" like anything because you can't "feel" at all if you are gone.
This argument is quite similar to the argument usually made for quantum immortality, where you can never experience death, since death is the cessation of all experiences. The difference here is that the quantum immortality argument is generalized to include other possible worlds (e.g. simulations of us in universes with different physical laws), not just Everett branches, and emphasis is given to resurrection simulations instead of "surviving branches".
Therefore, we can never truly "die".
On one hand, quantum immortality - probably yes. On the other hand, look at your observer-moments from a timeless perspective:
Instead of seeing time as something that is a part of you and stubbornly keeps crawling forward from every moment of your existence, imagine time as something that is already out there, just some scaffolding of the space-time, a way to cut existence into many small 4D cubes, some of which contain you, and most of them do not. From this perspective, death at time T simply means that most of your observer-moments happen before the time T, and only a few of them (because of quantum immortality) after the time T.
That is, the "feeling" of death in e.g. 2040 is simply that whenever you look at the calendar, you will almost certainly notice that it is not 2040 yet.
Seems to me that the correct conclusion, even after accepting quantum immortality, is not to imagine yourself in some weird unlikely future, but simply to... be in the present.
(The opposite is that you will exist for billions of years, experience tons of extremely unlikely things, like aliens, technological miracles, other kinds of magical resurrections, etc. Which would make the fact that you are right now in a normal situation astronomically unlikely. And yet, here you are.)
...of course this may all be bullshit and confused thinking; it's just currently the best I can do...
In the sense of "expect something happen over other things" you might not "ought" to "expect" to feel anything.
I am not convinced that memory-similarity is enough to get an "experience link". If there is right now a emulated mind running on a machine on the other side of the room, "I" do not feel "it's experience" even if our experiences happen to coincide in their structure (because it is an emulation of me).
But yeah dead minds do not do anything and that includes agony. Sometimes this is phrased as "Your death is not an event in your life". Whether something comes after is not terribly relevant or needed. I do not fully get how that there are going to be experiences in the future allows to deduce the quality of experiences now at all - I don't have paranormal ability to feel sleepy when the impending gap in conciouness is only going to be temporary. The end of yet is free to have any flavour it happens to have.
One of the things is that you would not expect your memory to be faithfull to what has actually happened to you. That is a simulation where you are booted with the lifememories of a rockstar is not any harder or no reason to think it is more propable than have the new entitys state pick up where you left off.
So you are in a different time, different place, with a different psychological structure. But you get to claim that it is "still you". There is not much left of what we use identity in daily life for in that case.
And to be consistent one would be adviced to "expect" at any moment to fluctuate into mars and choke for lack of air. Any kind of "amplitude anthropics" will need to trash the sense of the universe having reliable classical laws. Thus it might be more accurate to say that one can't expect anything. For each hopeful thread of the sprinkler there are going to be combinations of events that are not fun. That is being resurrected and living fine in the future has more than 0 probability but so does being resurrected and slaving away in a dystopia. And it is not like the universe would suddenly abandon its indifference to human desire just for respecting non-destruction.
I think the main crux here is the philosophy of identity. You do not regard the emulated mind running on a machine on the other side of the room as "you", but if the subjective experiences are identical, you cannot rule out the possibility of being the emulated mind yourself. They are functionally identical and thus should be both considered "you" as they are carrying out the same computation.
"And to be consistent one would be adviced to "expect" at any moment to fluctuate into mars and choke for lack of air."
You're right that this is a probability and in a multiverse this possibility would be realised. But in this case the probability of it occurring is so little we needn't pay attention to it. This is qualitatively different from "low probability of being resurrected", because "teleporting to Mars" has a counterfactual experience of "not teleporting to Mars" which outweighs its probability, whereas "experiencing resurrection" does not have a corresponding counterfactual experience of "experiencing non-experience" since this is a tautologically meaningless statement.
"And it is not like the universe would suddenly abandon its indifference to human desire just for respecting non-destruction."
I am not saying the universe necessarily has to care about human experiences at all, just that death and nonexperience is a tautologically meaningless concept. In fact I'd rather the universe did not make me immortal.
I know it is going to go into gnarly territority but I don't see how the functional identity has implications about the experiences. Lets say that the emulation is going to be perfect until it hits a float overflow point where there are going to be a slight difference. Before the divergence I can't know who I am. But when that divergence is observed I have resolved the indexical uncertainty. But it would seem that this kind of thing won't create two "streams of experiences" at that divergence point but it was rather two streams of experiences all along that can start to tell each other appart at that point.
To make the all the possibilities of every choice to be allways the same then details of our hardwares would need to keep the same. And if we are different enough that it makes sense to call one of us an emulation then that kind of difference will always exist.
If I am in a windowless room and I know that there are 300 such rooms on earth, I am still in one room instead of being in 300 of them (regardless of whether I know there are 299 identical humans in the other rooms). Somebody that only cared about a very superficial similarity could feel like that if a person that gets the same name as I had gets born that that is "identical enough" to call that they would be "me". Even complete data identicality does not get rid of having to take the differences into account so it is an improperly arbitrary identity.
"experiencing resurrection" has counterfactuals like "experiencing a coma", "experiencing a deep sleep", "experiencing an archeological limited-emulation interview","experiencing a lobotomized resurrection", etc. Taking so wide a definition that all those cases are included in "resurrection" makes it so vague that it doesn't provide much solace.
I disagree with your 300 room argument. My identity is tied to my mind, which is a computation carried out by all 300 copies of my body in these 300 rooms. If all 300 rooms were suddenly filled with sleeping gas and 299 of the copies are quietly killed, only 1 copy will wake up. However, I should expect to experience waking up with probability 1 since that is the only possible next observer moment in this set up. The 299 dead copies of me cannot generate any further observer moments since they are dead.
I'd argue that you cannot experience a coma since you're unconscious, you can't really experience anything when you're in a coma. When you wake up it will feels as if you have just jumped forward in time. Deep sleep is qualitatively different and I was careful to avoid saying it is exactly like sleep, since sleep is simply a state of lowered consciousness instead of complete unconsciousness (e.g. we can still experience things like dreams). It is possible coma is also lowered consciousness like sleep, in which case you can experience comas but this says nothing about experiencing unconsciousness.
Scenario to tease out where my intuition diverges
Alien looks at earth. With super-dyper tech they observe a lot of details and start predicting how one particular human will live. With super-dyper tech and galaxys worth of resources to model one planet they get good accuracy even if it is quite intensive deduction. Based on the prediction the aliens go to the predicted death event of that one particular human and copy the brain state to a flesh and blood body. They wake the reincarnation on their own planet and do interviews or whatever anthropology they set out to do (maybe even with galaxy-budjets there are resource limits and not having to simulate is economically significant).
Aliens couuld simulate faster than time ticks on earth and get a signifcant heada-up. When the body wakes up the "original" is separated by more-than-lightspeed difference. So there exists spacetime causality isolation.
When the person dies (on earth), if he would know about what the aliens did, should it give him solace? To my intuition there is no "carrying on" as there is no time connection between the death and the carnation.
Far future doing the resurrection is not useful for this point as your life would be a cause for the resurrection.
You seem to be arguing that you will experience "yourself" in many other parts of a multiverse after you die. Why does this not occur before you die?
Can you clarify what you mean by "feel like falling asleep"? I don't see why there would be even a moment of unconsciousness if your idea is true.
"You seem to be arguing that you will experience "yourself" in many other parts of a multiverse after you die. Why does this not occur before you die?"
Because even though "you" in the sense of a computation have multiple embeddings in the multiverse, the vast vast majority of them share the same subjective experience and are hence functionally the same being, you can't yourself distinguish between them. The difference is that while some of these embeddings end when they die, you will only experience the ones which continue on afterwards (since you can't experience nonexperience).
"Can you clarify what you mean by "feel like falling asleep"? I don't see why there would be even a moment of unconsciousness if your idea is true."
I meant that death just feels like a timeskip subjectively. And you're correct that there wouldn't be any moments of unconsciousness, in fact this is already true in real life. You can't experience unconsciousness, and when you wake up it subjectively feels like a jump-forward from where you left off.
Have you read / are you interested in reading Project Lawful? It eventually explores this topic in some depth—though mostly after a million words of other stuff.