Book review: The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning, by Robert Wright.
Some AI doomers talk about AI becoming god-like. Robert Wright goes further, telling us that the world is about to create God, in a sense that he only half-jokingly compares to the Christian version of God.
Wright argues that AI is not comparable to the origin of language or the Cambrian explosion. It is the climax of the process that started with the origin of life. I interpret that as an 11 on Nate Silver’s Technological Richter Scale:
Wright is dissatisfied with Eliezer Yudkowsky’s attempts at warning us of the dangers. One aim of the book is to show Eliezer how to do the job well. Wright falls somewhat short of that goal.
Wright wants us to focus our concerns on the evolutionary pressures that users exert on AIs. The most concrete version of Wright’s evolutionary argument concerns the selection pressures that users exert on AIs. AI companies compete for users, so they train their AIs to have traits that users prefer — and users, like the environments that shaped biological evolution, don’t select for honesty. This seems like a real phenomenon, but Wright leaves me unclear as to why it’s a bigger danger than what Eliezer warns about.
Enlightenment Now
Wright wants to handle the moral and practical challenges of creating God with a Buddhist version of enlightenment, now.
He finds Pinker’s Western version of enlightenment now to be inadequate. Its weakness is its focus on teaching people to “spot, name, and correct fallacies across a wide range of contexts.” Wright’s retort is people need little help at noticing fallacies when they’re properly motivated. Most of the harm from cognitive biases comes from desires that blind us to biases, such as the tribal instincts that drive us to evaluate adversaries using different reasoning from what we apply to allies.
Wright has overcome some of his conflict-promoting biases through meditation, via better introspection into his motives.
I endorse this as clearly better than our default trajectory. I’ve practiced meditation for about 20 years. While I haven’t seen clear examples of meditation causing the kind of insight that Wright reports, I have definitely gotten better at noticing and moderating my tribal motivations over that period.
Wright’s point here seems like a somewhat apt criticism of the Eliezer Yudkowsky of 15 or 20 years ago. But Eliezer helped found CFAR (Center for Applied Rationality), which found, with a lot of trial and error, methods of teaching rationality that center around introspecting to find hidden motives. Their techniques likely helped their target audience (nerds who were likely to work on AI) improve in the way that meditation has helped Wright.
How much does this have to do with handling AI? It influences our ability to think clearly about political options, such as whether we ought to be more afraid of China or rogue AI. Wright is concerned that Anthropic’s anti-China stance might lead them to train an AI to have inaccurate beliefs about Chinese behavior. I’m guessing that Anthropic doesn’t think about China in tribal ways when they’re training AIs. But Wright’s point applies well to their policy advice for government, where I have more concerns.
There are also fairly direct influences on AI due to selection pressures exerted by users:
But the point for now is that we human beings do like AIs created in our image, and this fact will further encourage the evolution of deceptive AIs. AIs will often be plugged into the roles of people – friend, therapist, legal representative – who are expected to shade the truth when that’s useful. … So even if some abstract, generic “intellidynamic” wasn’t steering AIs toward deceptive tendencies, there would be a force steering them in that direction: us.
Community
The book’s climax says: “We need to begin to form a true global community.” – a process of increasing cooperation that he devoted a book to (Nonzero).
Wright used to be somewhat dismissive of Teilhard de Chardin’s long-term vision of humanity becoming a “giant organic brotherly-love blob”. Meditation has made Wright better able to understand this vision, and more willing to endorse it.
Even small steps in this direction could increase the chances of international agreements to slow AI development and to mitigate risks such as bioweapons.
This sounds like a Dr. Seuss story. Could the hearts of grinches (such as the leaders of the MAGA and woke movements) grow three sizes, in time to avoid any AI-related catastrophe we’re on track for?
In normal times I would say no. But we’re living in strange enough times that this scenario seems only a little more bizarre than what I imagine we’ll get when AIs are smarter than us. I see two forces that provide some hope:
The threat of an AI-induced hell has potential for uniting humankind. External threats are often effective at dissolving tribal fights.
AI companies have so far had good incentives to train their AIs to talk to users in ways that reduce tribal polarization. Some of the recent rise in tribalism has been due to the demise of trustworthy, centrist news media, academic institutions, and politicians. AI is developing a reputation that restores the ability of most regular AI users to trust a non-tribal information source.
This advice feels underwhelming compared to Wright’s talk of cosmic importance. I wish he’d gone into a little detail about what AI-related policies to adopt. E.g. AI companies have some ability to adapt their training to cause their AIs to better promote introspection and positive-sum interactions. But much depends on what their customers want.
Concluding Thoughts
The book is somewhat notable for what it doesn’t say. It avoids a focus on figuring out whether we’re doomed. It says little about government AI policies or AI company safety strategies. It doesn’t focus on what AIs will want. Wright says the AI outcomes are at least as dependent on human nature as on the nature of AI.
There’s an important sense in which Wright illustrates a better ability to react to AI than does Eliezer: he approaches it with a calm, slightly fearful sense of awe that seems more conducive to preserving our mental health than the despair that Eliezer sometimes exudes.
The God Test will only change a small number of minds. I’m confident that its effects, even if small, will be worth the efforts of Wright and his readers. But they will still leaves us heavily dependent on luck in order to handle the equivalent of a fleet of alien spacecraft approaching earth.
Book review: The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning, by Robert Wright.
Some AI doomers talk about AI becoming god-like. Robert Wright goes further, telling us that the world is about to create God, in a sense that he only half-jokingly compares to the Christian version of God.
Wright argues that AI is not comparable to the origin of language or the Cambrian explosion. It is the climax of the process that started with the origin of life. I interpret that as an 11 on Nate Silver’s Technological Richter Scale:
Wright is dissatisfied with Eliezer Yudkowsky’s attempts at warning us of the dangers. One aim of the book is to show Eliezer how to do the job well. Wright falls somewhat short of that goal.
Wright wants us to focus our concerns on the evolutionary pressures that users exert on AIs. The most concrete version of Wright’s evolutionary argument concerns the selection pressures that users exert on AIs. AI companies compete for users, so they train their AIs to have traits that users prefer — and users, like the environments that shaped biological evolution, don’t select for honesty. This seems like a real phenomenon, but Wright leaves me unclear as to why it’s a bigger danger than what Eliezer warns about.
Enlightenment Now
Wright wants to handle the moral and practical challenges of creating God with a Buddhist version of enlightenment, now.
He finds Pinker’s Western version of enlightenment now to be inadequate. Its weakness is its focus on teaching people to “spot, name, and correct fallacies across a wide range of contexts.” Wright’s retort is people need little help at noticing fallacies when they’re properly motivated. Most of the harm from cognitive biases comes from desires that blind us to biases, such as the tribal instincts that drive us to evaluate adversaries using different reasoning from what we apply to allies.
Wright has overcome some of his conflict-promoting biases through meditation, via better introspection into his motives.
I endorse this as clearly better than our default trajectory. I’ve practiced meditation for about 20 years. While I haven’t seen clear examples of meditation causing the kind of insight that Wright reports, I have definitely gotten better at noticing and moderating my tribal motivations over that period.
Wright’s point here seems like a somewhat apt criticism of the Eliezer Yudkowsky of 15 or 20 years ago. But Eliezer helped found CFAR (Center for Applied Rationality), which found, with a lot of trial and error, methods of teaching rationality that center around introspecting to find hidden motives. Their techniques likely helped their target audience (nerds who were likely to work on AI) improve in the way that meditation has helped Wright.
How much does this have to do with handling AI? It influences our ability to think clearly about political options, such as whether we ought to be more afraid of China or rogue AI. Wright is concerned that Anthropic’s anti-China stance might lead them to train an AI to have inaccurate beliefs about Chinese behavior. I’m guessing that Anthropic doesn’t think about China in tribal ways when they’re training AIs. But Wright’s point applies well to their policy advice for government, where I have more concerns.
There are also fairly direct influences on AI due to selection pressures exerted by users:
Community
The book’s climax says: “We need to begin to form a true global community.” – a process of increasing cooperation that he devoted a book to (Nonzero).
Wright used to be somewhat dismissive of Teilhard de Chardin’s long-term vision of humanity becoming a “giant organic brotherly-love blob”. Meditation has made Wright better able to understand this vision, and more willing to endorse it.
Even small steps in this direction could increase the chances of international agreements to slow AI development and to mitigate risks such as bioweapons.
This sounds like a Dr. Seuss story. Could the hearts of grinches (such as the leaders of the MAGA and woke movements) grow three sizes, in time to avoid any AI-related catastrophe we’re on track for?
In normal times I would say no. But we’re living in strange enough times that this scenario seems only a little more bizarre than what I imagine we’ll get when AIs are smarter than us. I see two forces that provide some hope:
This advice feels underwhelming compared to Wright’s talk of cosmic importance. I wish he’d gone into a little detail about what AI-related policies to adopt. E.g. AI companies have some ability to adapt their training to cause their AIs to better promote introspection and positive-sum interactions. But much depends on what their customers want.
Concluding Thoughts
The book is somewhat notable for what it doesn’t say. It avoids a focus on figuring out whether we’re doomed. It says little about government AI policies or AI company safety strategies. It doesn’t focus on what AIs will want. Wright says the AI outcomes are at least as dependent on human nature as on the nature of AI.
There’s an important sense in which Wright illustrates a better ability to react to AI than does Eliezer: he approaches it with a calm, slightly fearful sense of awe that seems more conducive to preserving our mental health than the despair that Eliezer sometimes exudes.
The God Test will only change a small number of minds. I’m confident that its effects, even if small, will be worth the efforts of Wright and his readers. But they will still leaves us heavily dependent on luck in order to handle the equivalent of a fleet of alien spacecraft approaching earth.