I've been trying to convince my father to support the cause, and ran into resistance on a front that I didn't expect. It's hard to tell how much he's looking for an argument and how much he actually believes what he's advocating, but he doesn't display any behavior that would contradict him believing it and several of us (LauraABJ, SarahC and Andrew) were on hand and unable to shake him.
Today he emailed me these "Thoughts on Immortality"
Our not wanting to die is a bit of irrational behavior selected for by evolution.The universe doesn’t care if you’re there or not.The contrasting idea that you are the universe is mystical, not rational.
The idea that you are alive “now” but will be dead “later” is irrational. Time is just a persistent illusion according to relativistic physics.You are alive and dead, period.
A cyber-replica is not you. If one were made and stood next to you, you would still not consent to be shot.
Ditto a meat replica
If you believe the many worlds model of quantum physics is true (Eliezer does), then there already are a vitually infinite number of replicas of you already, so why bother making another one?
Given we'd already been over this several times I decided to try a different approach this tme, so this was my completely off-the-cuff reply:
"Are you here to have an argument? I'm sorry, this is abuse.
Terminal values and preferences are not rational or irrational. They simply are your preferences. I want a pizza. If I get a pizza, that won't make me consent to get shot. I still want a pizza. There are a virtually infinite number of me that DO have a pizza. I still want a pizza. The pizza from a certain point of view won't exist, and neither will I, by the time I get to eat some of it. I still want a pizza, damn it.
Of course, if you think all of that is irrational, then by all means don't order the pizza. More for me."
He's effectively an atheist so no need to worry about that angle. He would be a potentially strong asset were he to come around, and I hate to see him sit around without hope effectively waiting to die; when he tries to do good he doesn't exactly give to the Society for Cute Kittens and Rare Diseases but he's accomplishing far less than he could. I also would feel better knowing he fully supported my signing up for Alcor. More generally, I'd like to figure out how to pierce this sort of argument in a way that makes the person in question actually change his mind.
Your reasoning is correct, albeit simplified. Such a tradeoff is limited by the extent to which the older paperclip maximizer can be certain that the newer machine actually is a paperclip maximizer, so it must take on the subgoal of evaluating the reliability of this belief. However, there does exist a certainty threshold beyond which it will act as you describe.
Also, the paperclip maximizer uses a different conception of (the nearest concept to what humans mean by) "identity" -- it does not see the newer clippy as being a different being, so much as an extension of it"self". In a sense, a clippy identifies with every being to the extent that the being instantiates clippyness.
a clippy identifies with every being to the extent that the being instantiates clippyness.
But what constitutes 'clippyness'? In my comment above, I mentioned values, knowledge, and (legal?, social?) rights and obligations.
Clearly it seems that another agent cannot instantiate clippyness if its final values diverge from the archetypal Clippy. Value match is essential.
What about knowledge? To the extent that it is convenient, all agents with clippy values will want to share information. But if the agent instances are sufficiently distant, it is inevit... (read more)
Terminal values and preferences are not rational or irrational. They simply are your preferences. I want a pizza. If I get a pizza, that won't make me consent to get shot. I still want a pizza. There are a virtually infinite number of me that DO have a pizza. I still want a pizza. The pizza from a certain point of view won't exist, and neither will I, by the time I get to eat some of it. I still want a pizza, damn it.
Of course, if you think all of that is irrational, then by all means don't order the pizza. More for me."