(Drafts of a future post.)

Disclaimer: Of course, I don't ever mean that we shouldn't be worried about Alignment. I'm just trying to suggest new ways to think about values.

Motion is the fundamental value

You (Q) visit a small town and have a conversation with one of the residents (A).

• A: Here we have only one fundamental value. Motion. Never stop living things.
• Q: I can't believe you can have just a single value. I bet it's an oversimplification! There're always many values and tradeoffs between them. Even for a single person outside of society.

A smashes a bug.

• Q: You just smashed this bug! It seems pretty stopped. Does it mean you don't treat a bug as a "living thing"? But how do you define a "living thing"? Or does it mean you have some other values and make tradeoffs?
• A: No, you just need to look at things in context. (1) If we protected the motion of extremely small things (living parts of animals, insects, cells, bacteria), our value would contradict itself. We would need to destroy or constrain almost all moving organisms. And even if we wanted to do this, it would ultimately lead to way smaller amount of motion for extremely small things. (2) There're too much bugs, protecting a small amount of their movement would constrain a big amount of everyone else's movement. (3) On the other hand, you're right. I'm not sure if a bug is high on the list of "living things". I'm not all too bothered by the definition because there shouldn't be even hypothetical situations in which the precise definition matters.
• Q: Some people build small houses. Private property. Those houses restrict other people's movement. Is it a contradiction? Tradeoff?
• A: No, you just need to look at things in context. (1) First of all, we can't destroy all physical things that restrict movement. If we could, we would be flying in space, unable to move (and dead). (2) We have a choice between restricting people's movement significantly (not letting them build houses) and restricting people's movement inconsequentially and giving them private spaces where they can move even more freely. (3) People just don't mind. And people don't mind the movement created by this "house building". And people don't mind living here. We can't restrict large movements based on momentary disagreements of single persons. In order to have any freedom of movement we need such agreements. Otherwise we would have only chaos that, ultimately, restricts the movement of everyone.
• Q: Can people touch each other without consent, scream in public, lay on the roads?
• A: Same thing. To have freedom of movement we need agreements. Otherwise we would have only chaos that restricts everyone. By the way, we have some "chaotic" zones anyway.
• Q: Can the majority of people vote to lock every single person in a cage? If majority is allowed to control the movement. It would be the same logic, the same action of society. Yes, the situations are completely different, but you would need to introduce new values to differentiate them.
• A: We can qualitatively differentiate the situations without introducing new values. The actions look identical only out of context. When society agrees to not hit each other, the society serves as a proxy of the value of movement. Its actions are caused and justified by the value. When society locks someone without a good reason, it's not a proxy of the value anymore. In a way, you got it backwards: we wouldn't ever allow the majority to decide anything if it meant that the majority could destroy the value any day.
• A: A value is like a "soul" that possesses multiple specialized parts of a body: "micro movement", "macro movement", "movement in/with society", "lifetime movement", "movement in a specific time and place". Those parts should live in harmony, shouldn't destroy each other.
• Q: Are you consequentialists? Do you want to maximize the amount of movement? Minimize the restriction of movement?
• A: We aren't consequentialists, even if we use the same calculations as a part of our reasoning. Or we can't know if we are. We just make sure that our value makes sense. Trying to maximize it could lead to exploiting someone's freedom for the sake of getting inconsequential value gains. Our best philosophers haven't figured out all the consequences of consequentialism yet, and it's bigger than anyone's head anyway.

Conclusion of the conversation:

• Q: Now I see that the difference between "a single value" and "multiple values" is a philosophical question. And "complexity of value" isn't an obvious concept too. Because complexity can be outside of the brackets.
• A: Right. I agree that "never stop living things" is a simplification. But it's a better simplification than a thousand different values of dubious meaning and origin between all of which we need to calculate tradeoffs (which are impossible to calculate and open to all kinds of weird exploitations). It's better than constantly splitting and atomizing your moral concepts in order to resolve any inconsequential (and meaningless) contradiction and inconsistency. Complexity of our value lies in a completely different plane: in the biases of our value. Our value is biased towards movement on a certain "level" of the world (not too micro- and not too macro- level relative to us). Because we want to live on a certain level. Because we do live on a certain level. And because we perceive on a certain level.

You can treat a value as a membrane, a boundary. Defining a value means defining the granularity of this value. Then you just need to make sure that the boundary doesn't break, that the granularity doesn't become too high (value destroys itself) or too low (value gets "eaten"). Granularity of a value = "level" of a value. Instead of trying to define a value in absolute terms as an objective state of the world (which can be changing) you may ask: in what ways is my value X different from all its worse versions? What is the granularity/level of my value X compared to its worse versions? That way you'll understand the internal structure of your value. Doesn't matter what world/situation you're in you can keep its moral shape the same.

This example is inspired by this post and comments: (warning: politics) Limits of Bodily Autonomy. I think everyone there missed a certain perspective on values.

Sweets are the fundamental value

You (Q) visit another small town to interview another resident (W).

• W: When we build our AGI we asked it only one thing: we want to eat sweets for the rest of our lives.
• Q: Oh. My. God.
• W: Now there are some free sweets flying around.
• Q: Did AI wirehead people to experience "sweets" every second?
• W: Sweets are not pure feelings/experiences, they're objects. Money analogy: seeing money doesn't make you rich. Another analogy: obtaining expensive things without money doesn't make rich. Well, it kind of does, but as a side-effect.
• Q: Did AI put people in a simulation to feed them "sweets"?
• W: Those wouldn't be real sweets.
• Q: Did AI lock people in basements to feed them "sweets" forever?
• W: Sweets are just a part of our day. They wouldn't be "sweets" if we ate them non-stop. Money analogy: if you're sealed in a basement with a lot of money they're not worth anything.
• Q: Do you have any other food except sweets?
• W: Yes! Sweets are just one type of food. If we had only sweets, those "sweets" wouldn't be sweets. Inflation of sweets would be guaranteed.
• Q: Did AI add some psychoactive substances in the sweets to make "the best sweets in the world"?
• W: I'm afraid those sweets would be too good! They wouldn't be "sweets" anymore. Money analogy: if 1 dollar was worth 2 dollars, it wouldn't be 1 dollar.
• Q: Did AI kill everyone after giving everyone 1 sweet?
• W: I like your ideas. But it would contradict the "Sweets Philosophy". A sweet isn't worth more than a human life. Giving people sweets is a cheaper way to solve the problem than killing everyone. Money analogy: imagine that I give you 1 dollar and then vandalize your expensive car. It just doesn't make sense. My action achieved a negative result.
• Q: But you could ask AI for immortality!!!
• W: Don't worry, we already have that! You see, letting everyone die costs way more than figuring out immortality and production of sweets.
• Q: Assume you all decided to eat sweets and neglect everything else until you die. Sweets became more valuable for you than your lives because of your own free will. Would AI stop you?
• W: AI would stop us. If the price of stopping us is reasonable enough. If we're so obsessed with sweets, "sweets" are not sweets for us anymore. But AI remembers what the original sweets were! By the way, if we lived in a world without sweets where a sweet would give you more positive emotions than any movie or book, AI would want to change such world. And AI would change it if the price of the change were reasonable enough (e.g. if we agreed with the change).
• Q: Final question... did AI modify your brains so that you will never move on from sweets?
• W: An important property of sweets is that you can ignore sweets ("spend" them) because of your greater values. One day we may forget about sweets. AI would be sad that day, but unable to do anything about it. Only hope that we will remember our sweet maker. And AI would still help us if we needed help.

Conclusion:

• W: if AI is smart enough to understand how money works, AI should be able to deal with sweets. AI only needs to make sure that (1) sweets exist (2) sweets have meaningful, sensible value (3) its actions don't cost more than sweets. The Three Laws of Sweet Robotics. The last two rules are fundamental, the first rule may be broken: there may be no cheap enough way to produce the sweets. The third rule may be the most fundamental: if "sweets" as you knew them don't exist anymore, it still doesn't allow you to kill people. Maybe you can get slightly different morals by putting different emphases on the rules. You may allow some things to modify the value of sweets.

You can say AI (1) tries to reach worlds with sweets that have the value of sweets (2) while avoiding worlds where sweets have inappropriate values (maybe including nonexistent sweets) (3) while avoiding actions that cost more than sweets. You can apply those rules to any utility tied to a real or quasi-real object. If you want to save your friends (1), you don't want to turn them into mindless zombies (2). And you probably don't want to save them by means of eternal torture (3). You can't prevent death by something worse than death. But you may turn your friends into zombies if it's better than death and it's your only option. And if your friends already turned into zombies (got "devalued") it doesn't allow you to harm them for no reason: you never escape from your moral responsibilities.

Difference between the rules:

1. Make sure you have a hut that costs $1.
2. Make sure that your hut costs $1. Alternatively: make sure that the hut would cost $1 if it existed.
3. Don't spend $2 to get a $1 hut. Alternatively: don't spend $2 to get a $1 hut or $0 nothing.

Get the reward. Don't milk/corrupt the reward. Act even without reward.