I just had a thought. If Mary was presented with a red, a blue, and a green tile on a white background could she identify which was which without additional visual context clues like comparing them to her nails? If not, I would expect a p-zombie to have the same issue implying that that failure isn't to do with consciousness.
Depending on who you are talking to for-profit corporations is a good analogy for what is meant by "misaligned". You can then point out that those same organizations are likely to make AI with profit maximization in mind, and might skimp out on moral restraint in favor of being superhumanly good at PR.
Use that comparison with the wrong person and they'll call you a communist.
I want to add that the AI probably does not know it is misaligned for a while.
This sounds similar to the replication crisis, in terms of the incentivization issues.
Under unification wouldn't it make sense to consider ourselves to be every instance of our mind state? So there's no fact of the matter of what your surroundings are like until they effect your mind state. Similarly, every past and future that is compatible with your current mind state happened and will happen respectively.
This isn't the flu. America has had 318,000 deaths so far. That's ~8.5 years worth of flu deaths. One of those years was from the last 26 days. If the world had America's almost 1 death per 1,000 people mortality rate, that would be about 7.8 million deaths. There are 1.7 million deaths globally. That's 6 million people spared! And frankly America is in at least a half banked lockdown.
If your country has almost no cases, that isn't something to complain about. Mass graves would mean that your country had failed to the point that they were having difficulty manages all of the corpses. This point will vary country to country, but it is a lot harder for a first world country to hit that point than you seem to think.
This doesn't require faster than light signaling. If you and the copy are sent way with identical letters, that you open after crossing each other's event horizons. You learn want was packed with your clone when you open your letter. Which lets you predict what your clone will find.
Nothing here would require the event of your clone seeing the letter to affect you. You are affected by the initial set up. If the clone counterfactually saw something else, this wouldn't affect you according to SIA. It would require some assumptions about the setup to be wrong for that to happen to your clone though.
Another example would be if you learn a star that has crossed your cosmic event horizon was 100 solar masses, it's fair to infer that it will become a black hole and not a white dwarf.
Sorry this is so late. I haven't been on the site for a while. My last post was in reply to no interference always being better than fighting it out. Most of the character's seem to think that stopping the baby eaters has more utility than letting the superhappies do the same thing to us would cost.
The story brings up the possibility that, the disutility of the babyeaters might outweigh the utility of humanity. There's certainly nothing logically impossible about this.
Just ask which algorithm wins then. At least in these kinds of situations udt does better. The only downside is the algorithm has to check if it's in this kind of situation; it might not be worth practicing.