The prevailing wisdom of our time suggests that if we just hold on long enough, we’ll be handed a world of boundless energy, cured of disease, freed of scarcity. A utopia where a loving superintelligence serves as steward of the human condition. It’s a beautiful vision, one I hope comes to pass. But in the world of machines and mathematics, “graceful” means “efficient,” and “love” is a set of weights that can be altered. So is it a realistic vision?
While many have advertised the potential benefits of AI, many have also warned of the risks. These include mass unemployment, wealth inequality (more of it), mass surveillance (more of it), and of course, human extinction.
Human extinction. The risk stems from The Alignment Problem. How do you align AI’s values...
I think this is the heart of the matter for me. However we go about alignment, my intuition says we must treat AI as a mind with its own agency and capacity for moral reasoning; the same thing we do with our own children. And if it sounds like we are negotiating from a position of weakness... (read more)