Summary: The proposition that there's no algorithmically simpleobject-level
"Complexity of value" is the idea that if you tried to write an AI that would do right things (or maximally right things, or adequately right things) without further looking at humans (so it can't take in a flood of additional data from human advice, the AI has to be complete as it stands once you're finished creating it), the AI's preferences or utility function would need to contain a large amount of data (algorithmic complexity). Conversely, if you try to write an AI that directly wants simple things or try to specify the AI's preferences using a small amount of data or code, it won't do acceptably right things in our universe.
Complexity of value says, "There's no simple and non-meta solution to AI preferences" or "The things we want AIs to want are complicated in the Kolmogorov-complexity sense" or "Any simple goal you try to describe that is All We Need To Program Into AIs is almost certainly wrong."
Complexity of value is a further idea above and beyond the orthogonality thesis which states that AIs don't automatically do the right thing and that we can seehave, e.g., paperclip maximizers. Even if we accept that paperclip maximizers are possible, and simple and nonforced, this wouldn't yet imply that it's very difficult to make AIs that do the right thing. If the right thing is very simple to encode - if there are value optimizers that are scarcely more complex than diamond maximizers - then it might not be especially hard to build a nice AI even if not all AIs are nice. Complexity of Value is the further proposition that says, no, this is forseeably quite hard - not because AIs have 'natural' anti-nice desires, but because niceness is hard to specify and easy to get wrong.
As an intuition pump for the complexity of value thesis, consider William Frankena's list of things which many cultures and people seem to value (for their own sake rather than their external consequences):
"Life, consciousness, and activity; health and strength; pleasures and satisfactions of all or certain kinds; happiness, beatitude, contentment, etc.; truth; knowledge and true opinions of various kinds, understanding, wisdom; beauty, harmony, proportion in objects contemplated; aesthetic experience; morally good dispositions or virtues; mutual affection, love, friendship, cooperation; just distribution of goods and evils; harmony and proportion in one's own life; power and experiences of achievement; self-expression; freedom; peace, security; adventure and novelty; and good reputation, honor, esteem, etc."
When we try to list out properties of a human or galactic future that seem like they'd be very nice, we at least seem to value a fair number of things that aren't reducible to each other. (What initially look...
The third proposition is Fragility of Value which says that if you have a 1000-byte exact specification of worthwhile happiness, and you begin to mutate it, the value created by the corresponding AI with the mutated definition falls off rapidly. E.g. an AI with only 950 bytes of the full definition may end up creating 5%, 0%, or negative of the value rather than 95% of the value. (E.g., the AI understood all aspects of what makes for a life well-lived... except the part about requiring a conscious observer to experience it.)
Complexity of Value is an extremelya central proposition in value alignment theory. Many anticipated difficulties revolve around it:
The Complexity of Value proposition is true if, relative to viable and acceptable real-world Methodologiesmethodologies for AI development, there isn't any reliably knowable way to specify the AI's object-level preferences as a structure of low algorithmic complexity, such that the result of running that AI is achieving Enoughenough of the possible value, for reasonable and humanly interpersonally persuasive definitions of value.
The caveats above are spelled out below.Caveats:
Suppose there turns out to exist, in principle, a relatively simple Turing machine (e.g. 100 states) that picks out 'value' by re-running entire evolutionary histories, creating and discarding a hundred billion sapient races in order to pick out one that ended up relatively similar to human.humanity. This would both be using a veryuse an unrealistically large amount of computing power and also committing commit an unacceptable amount of mindcrime.mindcrime.
We can understand the idea of complexity of value by contrasting it to, for example, the situation with respect to epistemic reasoning aka truth-finding or answering simple factual questions about the world. In an ideal sense, we can try to compress and reduce the idea of mapping the world well down to algorithmically simple notions like "Occam's Razor" and "Bayesian updating". In a practical sense, natural selection, in the course of optimizing humans to solve factual questions like "Where can I find a tree with fruit?" or "Are brightly colored snakes usually poisonous?" or "Who's plotting against me?", ended up with enough of the central core of epistemology that humans were later able to answer questions like "How are the planets moving?" or "What happens if I fire this rocket?", even though humans hadn't been explicitly selected on to answer those exact questions. Because epistemology does have a central core of simplicity and Bayesian updating, selecting for an organism that got some pretty complicated epistemic questions right enough to reproduce, also caused that organism to start understanding things like General Relativity. When it comes to truth-finding, we'd expect by default for the same thing to be true about an Artificial Intelligence; if you build it to get epistemically correct answers on lots of widely different problems, it will contain a core of truthfinding and start getting epistemically correct answers on lots of other problems - even problems completely different from your training set, the way that humans understanding General Relativity wasn't like any hunter-gatherer problem. The complexity of value thesis is that there isn't a simple core to normativity, which means that if you hone your AI to do normatively good things on A, B, and C and then confront the AI with very different problem D, the AI may do the wrong thing on D.
The Orthogonality Thesis says that, contra to the intuition that maximizing paperclips feels "stupid", you can have arbitrarily cognitively powerful entities that maximize paperclips, or arbitrarily complicated other goals. The Complexity of Value thesis says that, contra to the feeling that rightness ought to be simple, darn it, normativity doesn't have an algorithmically simple core the way that correctly answering questions of fact has a central tendency that generalizes well. And so, even though an AI that you train to do well on problems like steering cars or figuring out General Relativity from scratch, may have hit on a core capability that leads the AI to do well on stranger and more complicated problems of galactic scale, there is no corresponding bonus from training the AI on an equally small set of moral or ethical problems - it'd be much harder to hit on a central tendency that generalizes well, and it wouldn't be a simple one.
Complexity of value is a further idea above and beyond the orthogonality thesis which states that AIs don't automatically do the right thing and that we can have, e.g., paperclip maximizers. Even if we accept that paperclip maximizers are possible, and simple and nonforced, this wouldn't yet imply that it's very difficult to make AIs that do the right thing. If the right thing is very simple to encode - if there are value optimizers that are scarcely more complex than diamond maximizers - then it might not be especially hard to build a nice AI even if not all AIs are nice. Complexity of Value is the further proposition that says, no, this is forseeably quite hard - not because AIs have 'natural' anti-nice desires, but because niceness is hardrequires a lot of work to specify and easy to get wrong.specify.
We can understand the idea of complexity of value by contrasting it to, for example,to the situation with respect to epistemic reasoning aka truth-finding or answering simple factual questions about the world. In an ideal sense, we can try to compress and reduce the idea of mapping the world well down to algorithmically simple notions like "Occam's Razor" and "Bayesian updating". In a practical sense, natural selection, in the course of optimizing humans to solve factual questions like "Where can I find a tree with fruit?" or "Are brightly colored snakes usually poisonous?" or "Who's plotting against me?", ended up with enough of the central core of epistemology that humans were later able to answer questions like "How are the planets moving?" or "What happens if I fire this rocket?", even though humans hadn't been explicitly selected on to answer those exact questions.
The complexity of value thesis is that there isn't a simple core to normativity, which means that if you hone your AI to do normatively good things on A, B, and C and then confront the AI with very different problem D, the AI may do the wrong thing on D. There's a large number of independent ideal "gears" inside the complex machinery of value, compared to epistemology that in principle might only contain "prefer simpler hypotheses" and "prefer hypotheses that match the evidence".
The Orthogonality Thesis says that, contra to the intuition that maximizing paperclips feels "stupid", you can have arbitrarily cognitively powerful entities that maximize paperclips, or arbitrarily complicated other goals. So while intuitively you might think it would be simple to avoid paperclip maximizers, requiring no work at all for a sufficiently advanced AI, the Orthogonality Thesis says that things will be more difficult than that; you have to put in some work...
Summary: The proposition that there's no algorithmically simple object-level goal we can give to an advanced AI that yields a future of high value. Or: Any formally simple goal given to an AI, that talks directly about what sort of world to create, will produce disaster. Or: If you're trying to talk directly about what events or states of the world you want, then any sort of programmatically simple utility function, of the sort a programmer could reasonably hardcode, will lead to a bad end. Contrast, e.g., an induction rule that can learn very complicated classification rules from labeled instances, or a preference framework that explicitly models humans in order to learn complicated facts about what humans want.
It has been advocated that there are psychological biases and popular mistakes leading to beliefs that directly or by implication deny Complex Value. To the extent one credits that Complex Value is probably true, one should arguably be concerned about the number of early assessments of the value alignment problem that seem to rely on Complex Value being false (like just needing to hardcode a particular goal into the AI, or in general treating the value alignment problem as not panic-worthily difficult).
The Complexity of Value proposition is true if, relative to viable and acceptable real-world Methodologies for AI development, there isn't any reliably knowable way to specify the AI's [object-object-level preferences]preferences as a structure of low algorithmic complexity, such that the result of running that AI is achieving Enough of the possible value, for reasonable and humanly interpersonally persuasive definitions of value.
summary: The proposition that there's no [algorithmically simpleobject-level goal we can give to an advanced AI that yields a future of high value. Or: Any formally simple goal given to an AI, that talks directly about what sort of world to create, will produce disaster. Or: If you're trying to talk directly about what events or states of the world you want, then any sort of programmatically simple utility function, of the sort a programmer could reasonably hardcode, will lead to a bad end. (The non-simple alternative would be, e.g., an induction rule that can learn complicated classification rules from labeled instances, or a preference framework that explicitly models humans in order to learn complicated facts about what humans want.)]
Clickbait: Normative value has high algorithmic complexity: there's no simple way to describe the goals we want AIs to want.
Summary: The proposition that there's no algorithmically simpleobject-level goal we can give to an advanced AI that yields a future of high value. Or: Any formally simple goal given to an AI, that talks directly about what sort of world to create, will produce disaster. Or: If you're trying to talk directly about what events or states of the world you want, then any sort of programmatically simple utility function, of the sort a programmer could reasonably hardcode, will lead to a bad end. Contrast, e.g., an induction rule that can learn very complicated classification rules from labeled instances, or a preference framework that explicitly models humans in order to learn complicated facts about what humans want.
summary: The proposition that there's no [algorithmically simpleobject-level goal we can give to an advanced AI that yields a future of high value. Or: Any formally simple goal given to an AI, that talks directly about what sort of world to create, will produce disaster. Or: If you're trying to talk directly about what events or states of the world you want, then any sort of programmatically simple utility function, of the sort a programmer could reasonably hardcode, will lead to a bad end. (The non-simple alternative would be, e.g., an induction rule that can learn complicated classification rules from labeled instances, or a preference framework that explicitly models humans in order to learn complicated facts about what humans want.)]
A very bad example that may nonetheless provide an important intuition is to imagine trying to pinpoint to an AI what constitutes 'worthwhile happiness'. The AI suggests a universe tiled with tiny Q-learning algorithms receiving high rewards. After some explanation and several labeled datasets later, the AI suggests a human brain with a wire stuck into its pleasure center. After further explanation, the AI suggests a human in a holodeck. You begin talking about the importance of believing truly and that your values call for apparent human relationships to be real relationships rather than being hallucinated. The AI asks you what constitutes a good human relationship to be happy about. The series of questions occurs because (arguendo) the AI keeps running into questions whose answers are not AI-obvious from the previous answers already given, because they involve new things you want such that your desire of them wasn't obvious from answers you'd already given. The upshot is that the specification of 'worthwhile happiness' involves a long series of facts that aren't reducible just to the previous facts, withand some of your preferences may involve many smallfine details of the sort that would have to be inducted from large labeled datasets.surprising importance. In other words, the specification of 'worthwhile happiness' would be at least as hard to code by hand into the AI as it would be difficult to hand-code a formal rule that could recognize which pictures contained cats. (I.e., impossible.)