2 Answers sorted by
60
I think its a mix of two (very related) things:
- General deep belief in moral anti-realism
- Ie, any given human has a set of values, and these values are not "special" from a objective standpoint. You have the values you have, and you follow them because they're your values, not because there is an external reason those values are "right"
- General deep belief in a weak form of orthogonality:
- Basically, we can imagine machines pursuing any random goal. As long as we can specify what that goal is, there's no obstacle to pointing it in that direction in principle.
It follows from these that solving alignment means we should be able to make the AI follow any random goal, and that "simplifying the problem" to only making it follow human values / human morality / the values of any given human, doesn't buy us much. It doesn't make the problem easier, because there's nothing special about human values.
I feel I'm not explaining myself very well. But imagine you wanna launch a rocket into the sun. The sun is a very distinguished and special object relative to earth. Probably your plan for launching the rocket into the sun will need to consider a bunch of details about the sun.
Now imagine instead you wanna launch your rocket to star 8285F171058B in the milky way. Probably most steps of that plan would be the same if you instead were sending the rocket to some different random star. This means that solving the problem of sending a rocket to a random star is a better line of attack than trying to analyse all the properties of star 8285F171058B. Most of those features will not be relevant to the hard part of the problem.
I feel quite a bit of skepticism over the idea that a consensus view of moral anti-realism would have led to a preference for an alignment framing.
For example, amongst non-experts, there is a strong consensus around what is moral and immoral conduct. Amongst moral philosophers, as I understand it, moral anti-realism is also a minority view. My understanding was that moral naturalism was closest to consensus. (Not to say moral anti-realism is necessarily wrong.) If there was some kind of article or post describing how this informed a shift in framing to al...
3
111
I personally avoid even using the words "morality" or "ethics" in the context of AI alignment, because both of those words reliably turn the vast majority of otherwise-sensible people into morons the moment they are spoken.
There is a basic question that has been confusing me for a while that I would like to ask about:
Why are the goals of AI safety, like achieving safety from extinction risks, or protection for human wellbeing, not more often framed as the goal of making moral machines? Or in other words, building AI that has a strong and reliable sense of morality and ethics.
There is definitely a lot of discussion around the edges of this question. For example, one recent post by @Richard_Ngo asked whether AI should be aligned to virtues. Or, a post from last year by @johnswentworth described thinking about what the alignment problem is. However, there's also a huge swath of writing where the concept of machine morality is never invoked or mentioned.
Part of the reason for my curiosity it that it seems like this framing could resolve a lot of confusion and in many ways it seems the most intuitive. For example, this seems like probably the most important framing that we apply, broadly, when trying to raise and educate safe and good humans.
This framing would also provide a nice way of synthesizing many different core AI safety results, like 'emergent misalignment.' We could simply say that AI exhibiting emergent misalignment did not possess a strong moral compass, or a strong sense of morality, prior to its fine-tuning.
Is there a kind of history with this framing where it was at some point made to seem outmoded or obsolete? I can imagine various obvious-ish objections, like the fact that morality is hard to define. (But again, the fact that this is the framing we run with humans makes it seem pretty powerful and flexible.) But it's not clear to me why this framing has any more or less issues than any other.
Greatly appreciate any input, or suggestions of where to look further.