Book review: Obsolete: The AI Industry’s Trillion Dollar Race to Replace Us? and How to Stop It, by Garrison Lovely.
Obsolete is a strange book. It’s mostly good, but the quality varies a good deal. Lovely is well informed about concerns that AI might kill us all, and doesn’t dispute those concerns. Yet he focuses much more on more ordinary concerns, in particular excessive concentration of power.
He is somewhat successful at portraying mundane harms of this year, and longer term risks (will AI kill us all in the 2030s), as similar problems. I usually cringe at such efforts, but Lovely is quite good at defusing the tensions between those two types of concern. E.g. he discusses AI-assisted political coups by a CEO such as Musk in a way that sounds almost realistic using current AI, while also warning about dramatic increases in that risk as AI becomes more powerful.
Most of the book is written for a pretty broad audience. I’m annoyed that he added in a modest number of gratuitous culture war quips that will repel some Republicans. He mitigates that a little by also complaining about widespread corruption among leading Democrats, but many Republicans won’t read far enough to notice that.
Obsolete also contains a few distracting sections on topics such as how to improve drug development. That passage would make an excellent blog post, but feels quite out of place in the middle of policy advice for dealing with AI risks.
Two examples of Lovely describing his concerns well:
He answers the “can’t you just unplug it” question by pointing to OpenAI’s inability to unplug a rogue CEO in November 2023.
He shows the Andy Jones chart of the horse population with a nearly 90% drop between 1930 and 1950.
For the first 120 years of that steady improvement, horses didn’t notice at all. … Progress in engines was steady. Equivalence to horses was sudden.
China
Lovely is correct to report that AI companies aren’t very good at keeping secrets, but he overstates how much that matters:
All of this means slowing US frontier software progress, via regulation or other means, wouldn’t hand China a lead it couldn’t otherwise achieve; it would slow the entire pack.
Copying US AI is not the main source of Chinese AI progress. This is an area with much disagreement among experts. Distillation is real, but overrated. My guess is that at least half of Chinese AI progress is due to there being plenty of low hanging fruit that many experts can find. Chinese companies face different constraints than US companies due to restrictions on selling compute to China. Chinese companies have adapted to their compute scarcity by focusing on performance optimizations that were lower priority for US companies. That means there were some medium-value respects in which China has led the US. See Aidan Gomez, Kyle Chan, and Min et alfor more thorough arguments.
On the other hand, Lovely underestimates how willing the Chinese government would be to slow an AI capabilities race if there were enough evidence to convince Trump to do the same. They’re not very eager to race toward superintelligence. Instead, they’re racing toward widespread deployment of AI applications such as robotics, which is where they have an advantage over the US.
AI’s China hawks – Altman and Amodei – are right about one thing: there is a titanic struggle over who controls the future. But it’s not between democracies and autocracies, it’s between humanity and them.
There are multiple struggles over AI. Lovely oversimplifies here. I want encourage more cooperative approaches to handling AI. I worry about Lovely encouraging conflict.
Antitrust
One of Lovely’s policy prescriptions starts with the great title “Too big to guardrail”. But he mostly wants to make AI companies smaller by using antitrust law.
Antitrust action to break up Nvidia or TSMC could make AI more affordable to the average person, more affordable to companies that want to replace human workers with AIs, and could help ensure that many nations have drone armies, rather than drone armies being concentrated in a smaller number of powerful nations. Those seem like mostly the opposite of what Lovely and I want.
Lovely seems more focused on breaking up Anthropic and OpenAI. But antitrust seems like the wrong tool here. Much of what concerns me about those two are the reckless competition between them that closely matches what antitrust law promotes.
Maybe Lovely has a good idea about how to make AI companies smaller, but he sure failed to communicate it here.
How Obsolete is the Book?
AI and the political landscape around it have changed frequently in the 6 months between when Lovely finished writing the book and publication. Most authors would have several claims about AI that got falsified in that time period. Roughly none of Lovely’s claims about what AIs do have been discredited, partly because he minimizes his reliance on claims about what AI can and can’t do.
Lovely has more mixed success at predicting what humans would do, but still looks pretty good.
If you model OpenAI as an organization trying to use AI to improve the world, you’ll be left confused by decisions like [releasing Sora]. But if you model it like Netflix or Meta – as another company trying to maximize its ownership share in all human attention – everything makes sense.
A month after Lovely finished writing the book, OpenAI shut down Sora. Apparently, owning much of our attention isn’t very valuable compared to what OpenAI is prioritizing now.
Anthropic’s alignment strategy simply wasn’t robust to a would-be despot taking the White House, even though it had already fucking happened.
This seems to refer to the Supply Chain Risk designation. I’d say Anthropic’s strategy worked as well as we could hope for, unlike some of its competitors.
Senator Bernie Sanders offered one answer in late 2025: a moratorium on data center construction in the US. In a world of half-measures, his proposal embodies the ambition needed to meet the moment.
Lovely proceeds in the next few paragraphs to mostly admit that the 2025 proposal was an inadequate half-measure. But he was partly prophetic, in that Sanders introduced a sufficiently ambitious bill just before Obsolete was published.
Lovely went out on a limb when he reported the beginnings of AI-related job losses, based on what was then rather thin evidence. Economists this month say the evidence has gotten stronger.
The tone of Obsolete seems optimized for public opinion as of when Lovely wrote it. Opinions are now more focused on loss of control:
Concluding Thoughts
This book will both increase understanding of why we should be concerned about AI, and push the US a little closer to turning AI into a tribal issue. I estimate an 80% chance that the book will have net positive effects.
More quotes:
When it’s deployed, the advanced agent is deliberately trying to capture the company that built it the way a corporation tries to capture a regulator.
Against standard inferences from the METR time horizon results:
Writing this book has been harder than all my other writing combined. As a piece gets longer, it gets exponentially harder to work on. … while much of the writing and reporting I’ve done over my career helped me write the book, many of the skills I needed to learn happened only over the course of writing the book itself.
Book review: Obsolete: The AI Industry’s Trillion Dollar Race to Replace Us? and How to Stop It, by Garrison Lovely.
Obsolete is a strange book. It’s mostly good, but the quality varies a good deal. Lovely is well informed about concerns that AI might kill us all, and doesn’t dispute those concerns. Yet he focuses much more on more ordinary concerns, in particular excessive concentration of power.
He is somewhat successful at portraying mundane harms of this year, and longer term risks (will AI kill us all in the 2030s), as similar problems. I usually cringe at such efforts, but Lovely is quite good at defusing the tensions between those two types of concern. E.g. he discusses AI-assisted political coups by a CEO such as Musk in a way that sounds almost realistic using current AI, while also warning about dramatic increases in that risk as AI becomes more powerful.
Most of the book is written for a pretty broad audience. I’m annoyed that he added in a modest number of gratuitous culture war quips that will repel some Republicans. He mitigates that a little by also complaining about widespread corruption among leading Democrats, but many Republicans won’t read far enough to notice that.
Obsolete also contains a few distracting sections on topics such as how to improve drug development. That passage would make an excellent blog post, but feels quite out of place in the middle of policy advice for dealing with AI risks.
Two examples of Lovely describing his concerns well:
He answers the “can’t you just unplug it” question by pointing to OpenAI’s inability to unplug a rogue CEO in November 2023.
He shows the Andy Jones chart of the horse population with a nearly 90% drop between 1930 and 1950.
China
Lovely is correct to report that AI companies aren’t very good at keeping secrets, but he overstates how much that matters:
Copying US AI is not the main source of Chinese AI progress. This is an area with much disagreement among experts. Distillation is real, but overrated. My guess is that at least half of Chinese AI progress is due to there being plenty of low hanging fruit that many experts can find. Chinese companies face different constraints than US companies due to restrictions on selling compute to China. Chinese companies have adapted to their compute scarcity by focusing on performance optimizations that were lower priority for US companies. That means there were some medium-value respects in which China has led the US. See Aidan Gomez, Kyle Chan, and Min et alfor more thorough arguments.
On the other hand, Lovely underestimates how willing the Chinese government would be to slow an AI capabilities race if there were enough evidence to convince Trump to do the same. They’re not very eager to race toward superintelligence. Instead, they’re racing toward widespread deployment of AI applications such as robotics, which is where they have an advantage over the US.
There are multiple struggles over AI. Lovely oversimplifies here. I want encourage more cooperative approaches to handling AI. I worry about Lovely encouraging conflict.
Antitrust
One of Lovely’s policy prescriptions starts with the great title “Too big to guardrail”. But he mostly wants to make AI companies smaller by using antitrust law.
Antitrust action to break up Nvidia or TSMC could make AI more affordable to the average person, more affordable to companies that want to replace human workers with AIs, and could help ensure that many nations have drone armies, rather than drone armies being concentrated in a smaller number of powerful nations. Those seem like mostly the opposite of what Lovely and I want.
Lovely seems more focused on breaking up Anthropic and OpenAI. But antitrust seems like the wrong tool here. Much of what concerns me about those two are the reckless competition between them that closely matches what antitrust law promotes.
Maybe Lovely has a good idea about how to make AI companies smaller, but he sure failed to communicate it here.
How Obsolete is the Book?
AI and the political landscape around it have changed frequently in the 6 months between when Lovely finished writing the book and publication. Most authors would have several claims about AI that got falsified in that time period. Roughly none of Lovely’s claims about what AIs do have been discredited, partly because he minimizes his reliance on claims about what AI can and can’t do.
Lovely has more mixed success at predicting what humans would do, but still looks pretty good.
A month after Lovely finished writing the book, OpenAI shut down Sora. Apparently, owning much of our attention isn’t very valuable compared to what OpenAI is prioritizing now.
This seems to refer to the Supply Chain Risk designation. I’d say Anthropic’s strategy worked as well as we could hope for, unlike some of its competitors.
Lovely proceeds in the next few paragraphs to mostly admit that the 2025 proposal was an inadequate half-measure. But he was partly prophetic, in that Sanders introduced a sufficiently ambitious bill just before Obsolete was published.
Lovely went out on a limb when he reported the beginnings of AI-related job losses, based on what was then rather thin evidence. Economists this month say the evidence has gotten stronger.
The tone of Obsolete seems optimized for public opinion as of when Lovely wrote it. Opinions are now more focused on loss of control:
Concluding Thoughts
This book will both increase understanding of why we should be concerned about AI, and push the US a little closer to turning AI into a tribal issue. I estimate an 80% chance that the book will have net positive effects.
More quotes:
Against standard inferences from the METR time horizon results: