Book review: The Infinity Machine: Demis Hassabis, DeepMind, and the
Quest for Superintelligence, by Sebastian Mallaby.
This book is a very good history of DeepMind.
Ender versus the pure scientist
An important theme of the book is that Hassabis identifies as pure
scientist, valuing knowledge for its own sake. He wants to surpass
Newton and Einstein at understanding reality.
He also identifies powerfully with Ender
Wiggin, wanting to play a
pivotal role in a critical interaction between two groups of
intelligences.
I see some important tension between these two goals. Are they
sufficiently similar that he can accomplish them both? Or is he dividing
his effort between them in a way that sabotages his chances of success?
Mallaby doesn't answer.
Mallaby documents some downsides to Hassabis's choice to emphasize
science over engineering.
One of the things that Sutskever loved about OpenAI was that it
revered engineers ... AI has academic roots, and academics tend to
look down on the dirty work of engineering. ... Hassabis and his
colleagues disparaged OpenAI's work as engineering-led: all brute
force and no intelligence.
OpenAI would have failed if it chose a "no intelligence" approach, but
underestimating brute force has been a more widespread mistake in AI
than underestimating intelligence. See the Bitter
Lesson.
DeepMind underestimated scaling laws circa 2019, and fell behind for a
while as a result.
Early Funding
DeepMind got adequate funding at its founding in 2010, but gradually
developed uncomfortable funding constraints that led to selling out to
Google in 2014.
Peter Thiel was comfortable investing in DeepMind in 2010, partly
because it was a clearly contrarian stance. But by late 2012, he
abandoned further investment because it was too expensive and
mainstream. I understand the too expensive part, but too mainstream in
2012 sounds bizarre.
Hassabis was unenthusiastic about joining Google. But he disliked
needing to spend time and effort on fundraising. His attitude in
resigning himself to cede significant control seems to reflect a
decision to prioritize being a scientist over being Ender.
Mallaby documents some later interactions with Google CEO Sundar Pichai
which indicate significant tension between Hassabis and Pichai around
2017, with Pichai sounding a bit myopic about the potential of AI.
Google's founders had a better grasp on the future of AI, but they
weren't active enough in the company to offset Pichai's nearer-term
focus.
Facebook offered more money than Google to buy DeepMind. Hassabis talked
with Zuckerberg, and rejected him on the grounds that he couldn't see
that AI was more important than virtual reality or 3D printing.
Elon Musk
Musk plays important roles in this story, laden with disturbing
contradictions.
He played a possibly important role in prompting Google to buy DeepMind,
by bragging to Larry Page about Musk's investment in DeepMind.
Musk recognized Hassabis's competence early on. But he switched to
disliking Hassabis in 2014 after trying to buy DeepMind and being
rejected in favor of Google.
Why did DeepMind reject Musk's bid? Mallaby doesn't say much here.
Other sources provide conflicting impressions as to which offer promised
more autonomy for DeepMind. Hassabis mainly decided on the basis that
Google credibly promised to provide enough compute. Without Thiel's
support, Musk's ability to raise enough money looked questionable. But
in hindsight, the examples of OpenAI and Anthropic indicate that it was
possible, but hard, to raise enough money as an independent company.
Musk has often expressed a fear of AI being controlled by a big
corporation. That's strange to read now that Musk has merged his AI
company into a trillion dollar conglomerate. Was some of Musk's concern
specific to Google? Musk had a strong negative reaction to Larry Page's
position that it was speciesist to worry about AI replacing humanity.
Hassabis hoped to manage the risks of powerful AI by getting the best AI
developers on a single team, sometimes likened to a Manhattan Project.
That didn't survive contact with the egos that AI attracted. From a
chapter aptly titled Out of Eden:
But to believers in the singleton vision, OpenAI's founding
represented the Fall: the moment when the serpent brought evil into
the garden ... Hassabis, ever practical, was also angry in a simpler
way. Musk and Hoffman had been invited to the SpaceX gathering [a
DeepMind safety board meeting] in good faith. They had sat through
the meeting, listened to DeepMind's plans, and then used what they
had heard to double-cross him.
... in early 2014, when Elon Musk tried to buy DeepMind, allegedly to
safeguard it for humanity. A year later, Musk remained bitter that his
bid had been spurned; if he couldn't be the one to build AI, he
wanted nobody to do so.
Musk ... continued to fulminate against DeepMind, denouncing Hassabis
as an evil genius, the evidence being that Hassabis had once worked on
a computer game called Evil Genius.
An ominous note:
in 2013, Elon Musk's wife, Talulah Riley - an actress known for
playing a seductive TV robot who takes to massacring humans
OpenAI
But Musk's fracturing of the AI industry wasn't the only obstacle to
cooperative AI development. Maybe the biggest setback came when OpenAI
released ChatGPT. AI companies at the time seemed to have a policy of
being pretty cautious about releasing new models. OpenAI changed their
policy to release ChatGPT rather hastily in response to a false rumor
that Anthropic was about to release the similar product that it had
created. Mallaby concludes:
Once ChatGPT had been embraced by consumers, the incentives for
gradualism crumbled.
Five months later, Hassabis told Mallaby:
This is wartime, OpenAI and Microsoft have literally parked the tanks
on the lawn.
Mustafa Suleyman
The book significantly raised my opinion of Suleyman. My review of his
book
was unenthusiastic.
Apparently he sounds a bit more forceful when talking to CEOs, and
consistently focused on some medium to large risks associated with AI.
He was the most eager of DeepMind leaders to pressure Google into
agreeing on an ethics and safety board.
Alas, he doesn't have the political skills to be effective at his
desired role.
In my review I dismissed his concern that "a jobs recession will crater
tax receipts". I recently examined this more
carefully,
and concluded that Suleyman's concern's are reasonable, and that
there's likely to be a few months or years of COVID-level stress before
government revenues become ample.
My Complaints
Most of the book seems carefully researched. Mallaby understands AI well
enough that I don't have any criticisms there.
Here are some flaws that annoyed me, but which don't detract much from
the book.
... Homo sapiens acquired the capacity for abstract thought, some
seventy thousand years ago
That's misleading at best. There's significant evidence of earlier
abstract thought. It seems likely that abstraction emerged gradually,
over a long period.
Mallaby wrote "Oxford's Future of Life Institute", when he meant to
refer to the Future of Humanity Institute.
In 2019, GPT-2 had barely been able to count up to five; it was
impressive in the same way that a four-year-old might be. In 2020,
GPT-3 was like a nine-year-old
I like attempts to explain AI progress by estimating the equivalent
human ages, but my experience indicates that AI is advancing at more
like two years of age per calendar year. Mallaby's estimates seem like
they're the result of cherry-picking the most impressive responses. I
focus somewhat on their planning abilities, whereas I doubt that Mallaby
puts any weight on those abilities. I say AI is just now reaching the
nine-year-old level.
Mallaby quotes Hinton as saying "There aren't any examples of more
intelligent things being controlled by less intelligent things".
While I approve of Mallaby's concerns about this kind of risk, he ought
to have pointed out that Hinton exaggerated. Some counter-examples:
Toxoplasma gondii controlling mice
Zombie Ant Fungus
The "pointy-haired boss" trope
Cats training humans to feed and pet them
The hunger-signaling part of the human mind overruling the part that
wants to lose weight
Concluding Thoughts
The book provides good insights into why AI became a race rather than a
Manhattan Project.
Mallaby leans toward the conclusion that a race was inevitable. I'm not
convinced.
This seems wrong:
The US-China race dynamic made it almost impossible to stanch the
intra-US race dynamic.
(I'm unsure whether Mallaby endorsed that view, or was merely reporting
Hassabis's view.) We know from nuclear non-proliferation treaties that
cooperation is possible between nations that are more hostile than the
current US-China conflict. This quote comes after Mallaby has devoted
plenty of analysis that points toward Altman and Musk triggering a race
for reasons that seem completely unrelated to China. I remain confused
as to why people treat China as anything more than a rationalization for
pursuing a competition that they want for less noble reasons.
The book convinced me that Hassabis is making a mistake by trying to be
both Einstein and Ender.
Hassabis has achieved partial success at Einstein-level science, while,
like Einstein, working at another job.
But being Ender is really a full-time job. It currently looks like
Hassabis is not narrowly enough focused on being Ender to be the pivotal
person in AI's destiny to remake the world.
Mallaby hints at Oppenheimer as a role model that might be appropriate
for Hassabis. Maybe the world would be better off if Hassabis had aimed
more for that role. Mallaby seems to say it's much too late to try
that.
More Quotes
"I think political systems will use it to terrorize people," Hinton
answered.
"Then why are you doing the research?" Bostrom asked.
"I could give you the usual arguments," Hinton replied. "But the
truth is that the prospect of discovery is too sweet."
According to David Silver:
When Demis talked about the influence of his mother and his horror of
manipulating others, he meant it. But it was one thing to abhor the
idea of controlling colleagues. Given his Jedi-level charisma, it was
quite another to avoid it.
I'm puzzled. I don't see signs of Jedi-level charisma.
Because of the black-box nature of these networks, the scientists who
built them often sounded like surprised parents. Look, my child can
say so many more words than just a week ago!
Hassabis was never part of the Singularity crowd. But he shared the
assumption that a "singleton" scenario provided the best shot at
safe AI ... He imagined convening a band of elite scientists in a
secluded research center, there to focus single-mindedly on the
birthing of safe superintelligence. This mash-up of Ender's
clandestine space station and the Manhattan Project's secret
encampment in New Mexico bubbled up in conversation periodically
Looks like the future of humanity is in a lot more danger because of an inebriated argument between Musk and Page. (The "speciesist" claim happened late at night at a party).
Ah, humans. Guys like them, particularly, I suppose.
Book review: The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence, by Sebastian Mallaby.
This book is a very good history of DeepMind.
Ender versus the pure scientist
An important theme of the book is that Hassabis identifies as pure scientist, valuing knowledge for its own sake. He wants to surpass Newton and Einstein at understanding reality.
He also identifies powerfully with Ender Wiggin, wanting to play a pivotal role in a critical interaction between two groups of intelligences.
I see some important tension between these two goals. Are they sufficiently similar that he can accomplish them both? Or is he dividing his effort between them in a way that sabotages his chances of success? Mallaby doesn't answer.
Mallaby documents some downsides to Hassabis's choice to emphasize science over engineering.
OpenAI would have failed if it chose a "no intelligence" approach, but underestimating brute force has been a more widespread mistake in AI than underestimating intelligence. See the Bitter Lesson.
DeepMind underestimated scaling laws circa 2019, and fell behind for a while as a result.
Early Funding
DeepMind got adequate funding at its founding in 2010, but gradually developed uncomfortable funding constraints that led to selling out to Google in 2014.
Peter Thiel was comfortable investing in DeepMind in 2010, partly because it was a clearly contrarian stance. But by late 2012, he abandoned further investment because it was too expensive and mainstream. I understand the too expensive part, but too mainstream in 2012 sounds bizarre.
Hassabis was unenthusiastic about joining Google. But he disliked needing to spend time and effort on fundraising. His attitude in resigning himself to cede significant control seems to reflect a decision to prioritize being a scientist over being Ender.
Mallaby documents some later interactions with Google CEO Sundar Pichai which indicate significant tension between Hassabis and Pichai around 2017, with Pichai sounding a bit myopic about the potential of AI. Google's founders had a better grasp on the future of AI, but they weren't active enough in the company to offset Pichai's nearer-term focus.
Facebook offered more money than Google to buy DeepMind. Hassabis talked with Zuckerberg, and rejected him on the grounds that he couldn't see that AI was more important than virtual reality or 3D printing.
Elon Musk
Musk plays important roles in this story, laden with disturbing contradictions.
He played a possibly important role in prompting Google to buy DeepMind, by bragging to Larry Page about Musk's investment in DeepMind.
Musk recognized Hassabis's competence early on. But he switched to disliking Hassabis in 2014 after trying to buy DeepMind and being rejected in favor of Google.
Why did DeepMind reject Musk's bid? Mallaby doesn't say much here. Other sources provide conflicting impressions as to which offer promised more autonomy for DeepMind. Hassabis mainly decided on the basis that Google credibly promised to provide enough compute. Without Thiel's support, Musk's ability to raise enough money looked questionable. But in hindsight, the examples of OpenAI and Anthropic indicate that it was possible, but hard, to raise enough money as an independent company.
Musk has often expressed a fear of AI being controlled by a big corporation. That's strange to read now that Musk has merged his AI company into a trillion dollar conglomerate. Was some of Musk's concern specific to Google? Musk had a strong negative reaction to Larry Page's position that it was speciesist to worry about AI replacing humanity.
Hassabis hoped to manage the risks of powerful AI by getting the best AI developers on a single team, sometimes likened to a Manhattan Project.
That didn't survive contact with the egos that AI attracted. From a chapter aptly titled Out of Eden:
An ominous note:
OpenAI
But Musk's fracturing of the AI industry wasn't the only obstacle to cooperative AI development. Maybe the biggest setback came when OpenAI released ChatGPT. AI companies at the time seemed to have a policy of being pretty cautious about releasing new models. OpenAI changed their policy to release ChatGPT rather hastily in response to a false rumor that Anthropic was about to release the similar product that it had created. Mallaby concludes:
Five months later, Hassabis told Mallaby:
Mustafa Suleyman
The book significantly raised my opinion of Suleyman. My review of his book was unenthusiastic.
Apparently he sounds a bit more forceful when talking to CEOs, and consistently focused on some medium to large risks associated with AI.
He was the most eager of DeepMind leaders to pressure Google into agreeing on an ethics and safety board.
Alas, he doesn't have the political skills to be effective at his desired role.
In my review I dismissed his concern that "a jobs recession will crater tax receipts". I recently examined this more carefully, and concluded that Suleyman's concern's are reasonable, and that there's likely to be a few months or years of COVID-level stress before government revenues become ample.
My Complaints
Most of the book seems carefully researched. Mallaby understands AI well enough that I don't have any criticisms there.
Here are some flaws that annoyed me, but which don't detract much from the book.
That's misleading at best. There's significant evidence of earlier abstract thought. It seems likely that abstraction emerged gradually, over a long period.
Mallaby wrote "Oxford's Future of Life Institute", when he meant to refer to the Future of Humanity Institute.
I like attempts to explain AI progress by estimating the equivalent human ages, but my experience indicates that AI is advancing at more like two years of age per calendar year. Mallaby's estimates seem like they're the result of cherry-picking the most impressive responses. I focus somewhat on their planning abilities, whereas I doubt that Mallaby puts any weight on those abilities. I say AI is just now reaching the nine-year-old level.
Mallaby quotes Hinton as saying "There aren't any examples of more intelligent things being controlled by less intelligent things".
While I approve of Mallaby's concerns about this kind of risk, he ought to have pointed out that Hinton exaggerated. Some counter-examples:
Concluding Thoughts
The book provides good insights into why AI became a race rather than a Manhattan Project.
Mallaby leans toward the conclusion that a race was inevitable. I'm not convinced.
This seems wrong:
(I'm unsure whether Mallaby endorsed that view, or was merely reporting Hassabis's view.) We know from nuclear non-proliferation treaties that cooperation is possible between nations that are more hostile than the current US-China conflict. This quote comes after Mallaby has devoted plenty of analysis that points toward Altman and Musk triggering a race for reasons that seem completely unrelated to China. I remain confused as to why people treat China as anything more than a rationalization for pursuing a competition that they want for less noble reasons.
The book convinced me that Hassabis is making a mistake by trying to be both Einstein and Ender.
Hassabis has achieved partial success at Einstein-level science, while, like Einstein, working at another job.
But being Ender is really a full-time job. It currently looks like Hassabis is not narrowly enough focused on being Ender to be the pivotal person in AI's destiny to remake the world.
Mallaby hints at Oppenheimer as a role model that might be appropriate for Hassabis. Maybe the world would be better off if Hassabis had aimed more for that role. Mallaby seems to say it's much too late to try that.
More Quotes
According to David Silver:
I'm puzzled. I don't see signs of Jedi-level charisma.