These are very preliminary notes, to get the rough ideas out. There's lots of research lying around, a paper in the works, and I'm happy to answer any and all questions. The Northstar of AI Alignment, as well as Alignment at Large, should be Superwisdom and Moral RSI (Recursive Self-Improvement)....
What if we expanded our lens beyond AI alignment, to grapple with the full scope of what "alignment" really means? We're facing not just the challenge of aligning artificial intelligence, but the deeper question of how to align technological capitalism itself — with its unconscious optimization pressures, its coordination failures,...
These are very preliminary notes, to get the rough ideas out. There's lots of research lying around, a paper in the works, and I'm happy to answer any and all questions.
The Northstar of AI Alignment, as well as Alignment at Large, should be Superwisdom and Moral RSI (Recursive Self-Improvement). Our current notion of human values is too shallow, too static, too corrupted.
Coherently Extrapolated Volition was directionally right — a method for continually extrapolating what we’d want to want if we were wiser, had grown up further, etc. However, this requires a non-arbitrary concept of wisdom and moral progress. I believe a developmentally informed Moral Realism can serve as the foundation for this:
It’s... (read 212 more words →)
What if we expanded our lens beyond AI alignment, to grapple with the full scope of what "alignment" really means? We're facing not just the challenge of aligning artificial intelligence, but the deeper question of how to align technological capitalism itself — with its unconscious optimization pressures, its coordination failures, its tendency toward extraction and nihilism — toward life-affirming possibilities.
Our world-system of technological capitalism is already a general, auto-poetic, and largely autonomous superintelligence.1 The Great Fact would lead us to believe that this system is aligned-ish: It has substantially raised living standards and life expectancy over the last two centuries.2 Unfortunately, this system is also misaligned with human and planetary flourishing in... (read 978 more words →)
Thanks! Yes, there's lots of convergence between methods, something Joe Carlsmith also wrote about. What do you see as the strongest arguments against Moral Realism?