1. Introduction

I will argue that the outcome we should aim to achieve from aligned AI is Hedonium: the conversion of the universe into the densest possible packing of positive valence. Valence represents the preferability of a given moment, as though we asked you to compare which moments of life you would like to live again and then numbered them such that you predictably preferred to live higher-numbered moments. Said another way, valence is a measure of happiness in its most exalted form, adjusting for the reductive implications of the word “happy.” Hedonium can be analogized to Computronium, but with computation replaced by positive valence. Thus, one image of what Hedonium might look like is a universe tiled with devices that simulate the experience of an enlightened monk at peak bliss. While Hedonium can seem initially unattractive, I argue the more one comes to understand it, the more one realizes it is aligned with their preferences and desires that it come to pass.

2. Preferences

This article will take a moral anti-realist perspective, meaning that when I argue that you should aim for Hedonium, I am arguing it is aligned with what you want. This angle is a bit slippery because it’s easy for such an argument to be dismissed along the following lines: “I know what I want better than anybody, and when I imagine Hedonium, I don’t want it; therefore, there is no argument that could convince me I want it.” As a response to this point, consider the following scenario: Jack is choosing between two piles of money. One is a large pile of $1 bills, and the other is a smaller pile of $100 bills; however, Jack is too far from the piles to make out the denominations of the bills.The $100 pile is worth more, but Jack chooses the larger pile because he assumes size tracks value. I will call the “error” he makes here value conflation (VC). He conflates the size of the money piles with their value and then chooses based on the size even though his intention is to maximize value. In this hypothetical, it’s a suboptimal but reasonable VC to make because Jack is bottlenecked on information that would help him evaluate the status of his terminal goal (maximizing total value) and so must use this intermediary. However, sometimes VC can be more subtle. Imagine Jack lives in a society that only has $1 bills such that the value of any money pile is always proportional to its si...

Posts

Wikitag Contributions

Comments