I didn't mean it literally. I meant, everything on which we base our long-term plans.

For example:

You go to school, save up money, try to get a good job, try to advance in your career... on the belief that you will find the results rewarding. However, this is pretty easily dismantled if you're not a life-extensionist and/or cryonicist (and don't believe in an afterlife). All it takes is for you to have the realization that

1) If your memory of an experience is erased thoroughly enough (and you don't have access to anything external that will have been alte... (read more)

First of all, I'm really glad we're having this conversation.

This question is the one philosophical issue that has been bugging me for several years. I read through your post and your comments and felt like someone was finally asking this question in a way that has a chance of being understood well enough to be resolved!

... then I began reading the replies, and it's a strange thing, the inferential distance is so great in some places that I also begin to lose the meaning of your original question, even though I have the very same question.

Taking a step bac... (read more)

6gjm7yWhy? If I have to choose between "happy for an hour, then memory-wiped" and "miserable for an hour, then memory-wiped" I unhesitatingly choose the former. Why should the fact that I won't remember it mean that there's no difference at all between the two? One of them involves someone being happy for an hour and the other someone being miserable for an hour. How so? Obviously my experience 100 years from now (i.e., no experience since I will most likely be very dead) will be the same as if I had never lived. But why on earth should what I care about now be determined by what I will be experiencing in 100 years? I don't understand this argument when I hear it from religious apologists ("Without our god everything is meaningless, because infinitely many years from now you will no longer exist! You need to derive all the meaning in your life from the whims of an alien superbeing!") and I don't understand it here either.
2Ishaan7yYes, under the assumption that you only value things that future-you will feel the effects of. If this is true, then all courses of action are equally rational and it doesn't matter what you do - you're at null. If you are such a being which values at least one thing that you will not directly experience, then the answer is no, these actions can still have worth. Most humans are like this, even if they don't realize it. Well...you'll still die eventually.

What makes us think _any_ of our terminal values aren't based on a misunderstanding of reality?

by bokov 1 min read25th Sep 201389 comments


Let's say Bob's terminal value is to travel back in time and ride a dinosaur.

It is instrumentally rational for Bob to study physics so he can learn how to build a time machine. As he learns more physics, Bob realizes that his terminal value is not only utterly impossible but meaningless. By definition, someone in Bob's past riding a dinosaur is not a future evolution of the present Bob.

There are a number of ways to create the subjective experience of having gone into the past and ridden a dinosaur. But to Bob, it's not the same because he wanted both the subjective experience and the knowledge that it corresponded to objective fact. Without the latter, he might as well have just watched a movie or played a video game.

So if we took the original, innocent-of-physics Bob and somehow calculated his coherent extrapolated volition, we would end up with a Bob who has given up on time travel. The original Bob would not want to be this Bob.

But, how do we know that _anything_ we value won't similarly dissolve under sufficiently thorough deconstruction? Let's suppose for a minute that all "human values" are dangling units; that everything we want is as possible and makes as much sense as wanting to hear the sound of blue or taste the flavor of a prime number. What is the rational course of action in such a situation?

PS: If your response resembles "keep attempting to XXX anyway", please explain what privileges XXX over any number of other alternatives other than your current preference. Are you using some kind of pre-commitment strategy to a subset of your current goals? Do you now wish you had used the same strategy to precommit to goals you had when you were a toddler?