Back in February, I attended the Bay Area EA Global as I have every year since they started having them. I didn't have a solid plan for what to do there this year, though, so I decided to volunteer. That means I only attended the sessions where I was on room duty, and otherwise spent the day having a few 1:1s when I wasn't on shift.
That's okay because, as everyone always says, the 1:1s are the best part of EA Global, and once again they were proven right.
Among the many great folks I met and friends I caught up with, I got the chance to meet Ronen Bar and learn about his idea of AI moral alignment. And when he told me about it, I was embarrassed I hadn't thought of it myself.
Simply put, moral alignment says that, rather than trying to align AI with human values, we try to explicitly align it to be a positive force for all sentient beings.
In all my years of thinking about AI alignment, I've not exactly ignored animals and other creatures both known and unknown, but I also figured they'd get brought along because humans care about them. But I have to admit, while it might come to the same outcome, it feels more authentic to say I want AI that is aligned to all beings rather than just humans because I, though I may be human, do in fact care about the wellbeing of all life and wish for all of it to flourish as it best can with the aid of future AI technology.
I think I missed articulating an idea like moral alignment because I was too close to the ideas. That is, I understood intuitively that if we succeeded in building AI aligned with human flourishing, that would necessarily mean alignment with the flourishing of all life, and in fact I've said that the goal of building aligned AI is to help life flourish, but not that AI should be aligned to all life directly. Now that we are much closer to building artificial superintelligence and need to figure out how to align it, the importance of aligning to non-human life stands out to me as a near-term priority.
For example, I can imagine us building human-aligned AI that ignores the plight of factory farmed animals, the suffering of shrimp, and the pain of bugs because lots of humans don't seem to care that much about their conditions. Such an AI would perhaps not be perfectly aligned in the ideal way we originally imagined aligned AI would be, but it would certainly be a kind of alignment with human goals, and it would be a travesty for the non-human beings it left out.
So let's not do that. Let's figure out how to align AI so that it's not just good for a few people or even all people, but so that it's good for all beings everywhere.
Partially covered this in my response to TAG above, but let me delve into that a bit more, since your comment makes a good point, and my definition of fairness above has some rhetorical dressing that is worth dropping for the sake of clarity.
I would define fairness at a high-level as - taking care not to gerrymander our values to achieve a specific outcome, and instead trying to generalize our own ethics into something that genuinely works for everyone and everything as best it can. In this specific case, that would be something along the lines of making sure that our moral reasoning is responsive first and foremost to evidence from reality, based on our best understanding of what kinds of metrics are ethically relevant.
For instance, what color a creature's shell, pelt or skin is has no significant ethical dimension, because it has limited bearing on things like suffering, pleasure and valence of that creature's experience (and what effect it does have is usually caused by a subjective aesthetic preference, often an externally-imposed one, rather than the color itself) - if our ethics considered exterior coloration as a parameter, our ethics would not in that case be built on a firm foundation.
Contrastingly, intelligence does seem to be a relevant ethical dimension because it determines things like whether an organism can worry about future suffering (thereby suffering twice), and whether an organism is capable of participating in more complex activities with more complex, potentially positive, potentially negative valences. Of course there is a great deal of further work required to understand how best to consider and parameterize intelligence for this context, but we are not unjustified in believing it is relevant.
I agree that ultimately choices are going to need to be made - I am of the opinion those choices should be as inclusive as possible, balancing against our best understanding of reality, ethics, and what will bring about the better outcome for all involved. Does that answer your question?
Back in February, I attended the Bay Area EA Global as I have every year since they started having them. I didn't have a solid plan for what to do there this year, though, so I decided to volunteer. That means I only attended the sessions where I was on room duty, and otherwise spent the day having a few 1:1s when I wasn't on shift.
That's okay because, as everyone always says, the 1:1s are the best part of EA Global, and once again they were proven right.
Among the many great folks I met and friends I caught up with, I got the chance to meet Ronen Bar and learn about his idea of AI moral alignment. And when he told me about it, I was embarrassed I hadn't thought of it myself.
Simply put, moral alignment says that, rather than trying to align AI with human values, we try to explicitly align it to be a positive force for all sentient beings.
In all my years of thinking about AI alignment, I've not exactly ignored animals and other creatures both known and unknown, but I also figured they'd get brought along because humans care about them. But I have to admit, while it might come to the same outcome, it feels more authentic to say I want AI that is aligned to all beings rather than just humans because I, though I may be human, do in fact care about the wellbeing of all life and wish for all of it to flourish as it best can with the aid of future AI technology.
I think I missed articulating an idea like moral alignment because I was too close to the ideas. That is, I understood intuitively that if we succeeded in building AI aligned with human flourishing, that would necessarily mean alignment with the flourishing of all life, and in fact I've said that the goal of building aligned AI is to help life flourish, but not that AI should be aligned to all life directly. Now that we are much closer to building artificial superintelligence and need to figure out how to align it, the importance of aligning to non-human life stands out to me as a near-term priority.
For example, I can imagine us building human-aligned AI that ignores the plight of factory farmed animals, the suffering of shrimp, and the pain of bugs because lots of humans don't seem to care that much about their conditions. Such an AI would perhaps not be perfectly aligned in the ideal way we originally imagined aligned AI would be, but it would certainly be a kind of alignment with human goals, and it would be a travesty for the non-human beings it left out.
So let's not do that. Let's figure out how to align AI so that it's not just good for a few people or even all people, but so that it's good for all beings everywhere.
Cross-posted from my blog, Uncertain Updates.