Imagine an astronomer who discovers an asteroid with a 50% chance of hitting Earth in 2035. She goes on TV. She testifies before Congress. She founds the Asteroid Deflection Institute and starts doing fundraising rounds. And then, in between appearances, she prolifically tweets about zoning reform.
I think that most people agree that this would be pretty weird. Her zoning takes might even be correct, but why would you do that in the first place? If she has attention left over for zoning, how real can the asteroid be? Thus, the asteroid stops being filed under "horrifying, world-uniting emergency" and starts being filed under "political cause".
This essay is obviously about AI safety. If you believe that AI is the biggest existential risk that humanity is facing in the 21st century, then you believe that you have located the asteroid. Almost everything else you care about is downstream of the asteroid not hitting. And yet the AI safety movement is full of people who profess this belief but also produce a steady stream of public commentary on housing policy, (unrelated) geopolitics, campus culture, and whatever else drifted across the timeline that week. If you don't believe me, just look at leading AI safety figures' accounts on Twitter.
I think that this is a mistake, and I want to explain why.
First of all, I don't want to argue that AI safety should stay out of politics. That would be stupid: AI safety is politics. You can't "stay out of politics" while asking governments to regulate compute, control exports, impose liability on the richest companies on Earth, and sign treaties with geopolitical rivals. An AI safety movement that tried to stay out of politics would consist of people writing alignment papers nobody reads.
What I am arguing is that AI safety should stay out of non-load-bearing politics (from now on called "hobby politics" for simplicity). Load-bearing politics is the stuff you can't avoid because it is literally what you came here for: compute governance, licensing, treaties, and so on. Hobby politics is everything else — your takes on the Middle East, on immigration, feminism, whether San Francisco should build more housing, and so on.
The load-bearing politics is the mission. The hobby politics is a tax on the mission, paid from the "credibility with people who don't already agree with us" account that you share with the rest of the movement.
So here is my thesis: AI safety should do the politics it must do to achieve its central aims (like "humanity doesn't die" and "humanity is not fully disempowered") and not one take more. In other words: AI safety cannot afford a second cause.
But surely this is not needed? After all, AI safety can avoid polarization by framing itself around universal values. Surely, "we don't want everyone to die" polls pretty well across most demographics these days.
This sounds compelling until you look at what happened to climate change.
Climate change had the most universalist framing imaginable. The planet is on fire, we need to save it to protect the future of our children. For God's sake, they had polar bears. But the polar bears didn't save them from polarization.
Why? First, because the solutions threatened specific interests: carbon taxes and drilling restrictions would hit certain industries, and so those industries spent decades and billions deliberately coding climate as a left-wing hobbyhorse. A polarized issue is a stalled issue, after all. Second, the proposed solutions happened to rhyme with the left tribe's preexisting preferences like regulation, international coordination, constraints on industry, so the climate change movement was nicely absorbed into the greater "left vs. right" tribal fistfight.
And the movement was not nearly disciplined enough to avoid this. Instead, climate change activists weighed in on every cause du jour imaginable. So persuadable-but-skeptical Republicans got confirmation of the suspicion the fossil lobby spent decades planting: this was never about the climate; this is a package deal, and it is a package I dislike.
The lesson from climate change is not "universal framing keeps you safe, unless someone is undisciplined." The lesson is: bipartisanship is an unstable equilibrium that hostile actors will actively try to push you out of, and maintaining that equilibrium will require deliberate, continuous effort. It will require cultivating champions across the aisle. It will also require framing that gives each tribe a native reason to care. And it will require not handing your prospective opponents ammunition for free.
We should keep in mind that the rationalist-adjacent / AI safety community is selected for people who love to argue. Debating is fun. Asking a rationalist to see a wrong opinion on the internet and not correct it is like asking a border collie to watch sheep walk by. The discipline requested is real discipline, and the movement has, let's say, an uneven track record with impulse control around Being Right in Public. But the virtue of silence is a real virtue.
And AI safety's asks rhyme uncomfortably with the existing package: "regulate tech companies" sounds left, "beat China" sounds right, and both tribes have noticed. Already, AI safety is starting to be seen as kind of left-coded (if you really think that the Grey tribe is a thing in broader politics, you are severely mistaken regarding people's ability and will to distinguish the Grey tribe from the Blue tribe). This could become a big problem given that Republicans currently hold power, and that even if they lose the next election, their support will matter a great deal going forward.
How do we avoid this kind of polarization? At the very least, we will need to stick to three simple rules:
Formalized organizational nonpartisanship. Safety orgs should adopt the standard think-tank posture: no endorsements (except for perhaps AI-safety-related ones), no institutional opinions on anything that isn't the asteroid.
A cultural norm that the more public-facing your safety role, the more your pet causes stay in the drawer.
Active and deliberate cultivation of champions on both the left and the right, because universalist framing alone won't do it for you.
This will not be easy, but I think it is very much worth doing to avoid AI safety going the way of climate change.
Imagine an astronomer who discovers an asteroid with a 50% chance of hitting Earth in 2035. She goes on TV. She testifies before Congress. She founds the Asteroid Deflection Institute and starts doing fundraising rounds. And then, in between appearances, she prolifically tweets about zoning reform.
I think that most people agree that this would be pretty weird. Her zoning takes might even be correct, but why would you do that in the first place? If she has attention left over for zoning, how real can the asteroid be? Thus, the asteroid stops being filed under "horrifying, world-uniting emergency" and starts being filed under "political cause".
This essay is obviously about AI safety. If you believe that AI is the biggest existential risk that humanity is facing in the 21st century, then you believe that you have located the asteroid. Almost everything else you care about is downstream of the asteroid not hitting. And yet the AI safety movement is full of people who profess this belief but also produce a steady stream of public commentary on housing policy, (unrelated) geopolitics, campus culture, and whatever else drifted across the timeline that week. If you don't believe me, just look at leading AI safety figures' accounts on Twitter.
I think that this is a mistake, and I want to explain why.
First of all, I don't want to argue that AI safety should stay out of politics. That would be stupid: AI safety is politics. You can't "stay out of politics" while asking governments to regulate compute, control exports, impose liability on the richest companies on Earth, and sign treaties with geopolitical rivals. An AI safety movement that tried to stay out of politics would consist of people writing alignment papers nobody reads.
What I am arguing is that AI safety should stay out of non-load-bearing politics (from now on called "hobby politics" for simplicity). Load-bearing politics is the stuff you can't avoid because it is literally what you came here for: compute governance, licensing, treaties, and so on. Hobby politics is everything else — your takes on the Middle East, on immigration, feminism, whether San Francisco should build more housing, and so on.
The load-bearing politics is the mission. The hobby politics is a tax on the mission, paid from the "credibility with people who don't already agree with us" account that you share with the rest of the movement.
So here is my thesis: AI safety should do the politics it must do to achieve its central aims (like "humanity doesn't die" and "humanity is not fully disempowered") and not one take more. In other words: AI safety cannot afford a second cause.
But surely this is not needed? After all, AI safety can avoid polarization by framing itself around universal values. Surely, "we don't want everyone to die" polls pretty well across most demographics these days.
This sounds compelling until you look at what happened to climate change.
Climate change had the most universalist framing imaginable. The planet is on fire, we need to save it to protect the future of our children. For God's sake, they had polar bears. But the polar bears didn't save them from polarization.
Why? First, because the solutions threatened specific interests: carbon taxes and drilling restrictions would hit certain industries, and so those industries spent decades and billions deliberately coding climate as a left-wing hobbyhorse. A polarized issue is a stalled issue, after all. Second, the proposed solutions happened to rhyme with the left tribe's preexisting preferences like regulation, international coordination, constraints on industry, so the climate change movement was nicely absorbed into the greater "left vs. right" tribal fistfight.
And the movement was not nearly disciplined enough to avoid this. Instead, climate change activists weighed in on every cause du jour imaginable. So persuadable-but-skeptical Republicans got confirmation of the suspicion the fossil lobby spent decades planting: this was never about the climate; this is a package deal, and it is a package I dislike.
The lesson from climate change is not "universal framing keeps you safe, unless someone is undisciplined." The lesson is: bipartisanship is an unstable equilibrium that hostile actors will actively try to push you out of, and maintaining that equilibrium will require deliberate, continuous effort. It will require cultivating champions across the aisle. It will also require framing that gives each tribe a native reason to care. And it will require not handing your prospective opponents ammunition for free.
We should keep in mind that the rationalist-adjacent / AI safety community is selected for people who love to argue. Debating is fun. Asking a rationalist to see a wrong opinion on the internet and not correct it is like asking a border collie to watch sheep walk by. The discipline requested is real discipline, and the movement has, let's say, an uneven track record with impulse control around Being Right in Public. But the virtue of silence is a real virtue.
And AI safety's asks rhyme uncomfortably with the existing package: "regulate tech companies" sounds left, "beat China" sounds right, and both tribes have noticed. Already, AI safety is starting to be seen as kind of left-coded (if you really think that the Grey tribe is a thing in broader politics, you are severely mistaken regarding people's ability and will to distinguish the Grey tribe from the Blue tribe). This could become a big problem given that Republicans currently hold power, and that even if they lose the next election, their support will matter a great deal going forward.
How do we avoid this kind of polarization? At the very least, we will need to stick to three simple rules:
This will not be easy, but I think it is very much worth doing to avoid AI safety going the way of climate change.