I wrote a substack article about how Anthropic is not behaving sensibly according to the worldview expressed by its leaders; about how Anthropic comms is slimy; and about what it might look like if they were living up to their own ideals. It includes embedded video clips to drive the point home, and I don't know how to add those here, so I'll mostly leave this as a linkpost. Below are some excerpts around the main arguments:
Is it moral to build a technology that threatens everyone’s life, if you think you can do the job safer than the competition? Does it change the moral equation if you get paid billions of dollars along the way? I don’t have a confident answer about what the right thing to do is under these dire circumstances. But I am confident of this: if you wish to take the responsibility of building superintelligence onto your own shoulders, it comes with an even greater responsibility to do everything in your power to change the circumstances.
[...]
When asked by Anderson Cooper in September, “Do you earnestly believe that AI could kill all humans?”, Dario changed the topic to insisting that Jacob Coxon (who had recently resigned) wasn’t primarily expressing concerns about Dario’s organization (a claim disavowed by Coxon almost immediately). The closest Dario comes to answering Anderson’s question is to claim that the “chance that things go wrong” is “not all that high”, which seems to fly in the face of Dario’s statements elsewhere.
[...]
Imagine that, when Anderson Cooper asked “Do you earnestly believe that AI could kill all humans?”, Dario answered:
Yes. Yes, I’ve been trying to raise the alarm about this for a decade, and I’m relieved that people are finally listening. I think that AI could bring some great benefits if we navigate it well, but if we navigate it poorly then it really does risk killing every human on the planet. I feel like we’re kind of trapped in a race here, where we at Anthropic think we can do it more safely than our other competitors, both domestic and abroad. But it’s an insane race, because we’re taking on an incredible amount of risk. We’re taking on far, far more risk than NASA takes on for a crewed shuttle flight, except instead of risking seven volunteers, we’re risking the entirety of humanity.
However, the abstract and distant nature of long-term risks makes them hard to approach from a policy perspective: our view is that it may be best to approach them indirectly by addressing more imminent risks that serve as practice for them.
This was about three years ago. Maybe he thought that the risks were long-term and distant in 2023? Nope! Fourteen days later, he told the podcaster Dwarkesh that the risk might be as little as 2-3 years away, adding:
[T]here will be powerful models; they will be agentic; we’re getting towards them. If such a model wanted to wreak havoc and destroy humanity or whatever I think we have basically no ability to stop it.
[...]
This is not what it would look like if the companies were actually trying to stop the reckless race.
[...]
What would it look like, if they were actually trying? Occasionally, I find myself in conversation with someone in a leadership position at a frontier AI company. Sometimes, when I express that I think they’re shirking their responsibilities, they ask what I think they could do. Here’s a list based on the one I was sending around to the labs this summer (updated lightly in wake of recent events):
Improve safety and security practices:
Clearly, loudly, and publicly state what would cause you to halt AI R&D.
Release as much information to the public as possible, so that there is full transparency about how fast the field is developing, and about how well AI alignment is going.
Make detailed, concrete, candid plans for aligning superintelligence and make them public, or make it clear that there is no plan.
Create a communications team at your organization that honestly communicates with the public. It should be one of the higher priorities of your organization to have a full-time team clearly stating things like, “The dangers look drastically higher than any engineer would accept on a bridge, or than NASA would accept on a crewed rocket launch. This is really bad, because it threatens all of civilization.” Treat cases like Coxon’s resignation as an opportunity to alert the world to the grave nature of the stakes.
Strongly encourage your employees to speak up about their actual views incessantly. Give employees extensive leeway to appear on podcasts talk about how crazy it is that the world is rolling the dice with all of our lives. If you wouldn’t push a button to cripple a large fraction of the scientific community’s ability to have an honest public discussion here, then don’t push that button for your own employees.
Create a prominent website banner warning the public. If you think you’re building machines that threaten the lives of your family and your world, you should be at least as loud and disruptive about this as Wikipedia is willing to be with a routine fundraiser. To do anything less sends a strong signal that you aren’t earnest about any of your talk of risk, that it isn’t truly urgent or worth even minor costs/distractions to your users, that recent AI progress is business-as-usual, etc.
Signal boost the worries of your researchers, and make it clear that to a significant extent we’re flying blind and rolling the dice with humanity’s future. If Evan Hubinger or Jan Leike or Paul Christiano or roon are saying things that would shock policymakers to their core, then the CEO should also be saying those exact same things. If the CEO disagrees with the many researchers who think we may be about to lose the future, then he should at least give these views top billing in quotation marks, and say why he disagrees.
Make a candid press statement, saying “This is the most dangerous thing humanity has ever done; the world should be going far more slowly; we wish we didn’t have to race, and we feel trapped. An international moratorium is warranted on a timescale of weeks, not years.”
Candidly brief Congress in public, begging for an urgent US-China deal to halt the race.
Tour other countries to raise an alarm. This should include costly stunts like getting on your knees and begging the UN to do something about the problem urgently, saying this is an ongoing emergency, or other things that break people out of the sense that this is all business as usual.
Be loud and candid about the fact that you weren’t being fully candid previously:
Acknowledge and apologize for your company’s past pattern of softpedaling the risk, and publicly commit to doing otherwise going forward. When a massive pattern of misleading statements over the years is never acknowledged, it becomes harder to believe that an organization has turned over a new leaf. Provide a long list of the many concrete missteps, such as this one (where the new Anthropic Institute created to “advance the public conversation about AI” said that AI “may also create new challenges, including abrupt economic changes and broad societal impacts”, which is a funny way of acknowledging that leadership thinks AI could extinguish humanity) or this one (where discussion of AI risk was turned into a literal ad for Claude).
Emphasize the sharpness of the change, if you can do so honestly. Communicate that you’re coming clean, to the extent that you really are and to the extent that that’s possible. Replacing senior staff who were involved in past communications blunders would be a great way to credibly signal that the change isn’t just rhetoric. Being concrete and loud about the mistakes also helps (e.g., “the situation has gotten a lot more urgent” and/or “now that we’ve see the consequences, we think the old approach was counterproductive”). Do postmortems and root cause analysis and publish those. (See also Nathan Calvin pointing out that Anthropic doesn’t seem to be doing much root cause analysis of how their lobbyists wound up pushing weak, self-serving draft legislation.)
Support helpful policy responses:
Actively fight against efforts to left-code or polarize this issue. Be willing to lose some important Democratic allies for the sake of building a coalition with both parties.
Put a huge amount of funding into things like tamper-proof location verification technology and tamper-proof monitoring technology on advanced AI chips, alongside lobbying for international slowdown.
AI companies haven’t done any of this stuff. Wikipedia spends more effort getting you to donate $2 every year than AI companies spend trying to alert the world to the fact that they’re gambling with your life.
It might be okay to gamble with everyone’s life if that’s the only way to protect them from the next guy making a worse gamble. But that excuse only works if you’re also trying with all your might to get the whole gamble called off. The AI companies are not living up to that standard.
[...]
Anthropic is not trying to stop the AI race with all its might. So we’ll just have to stop it without them.
Here's another convenience link to the full article, including video clips of the relevant excerpts.
Thank you. Anthropic may be a tragedy, but it was an entirely predictable tragedy.
Moral corruption in the face of money, power and "being an insider" is entirely unsurprising. Anyone who did not expect Dario and Anthropic to wind up here should update.
You should also generalize this principle to the organizations working on "insider" reform, and to "insider" oversight organizations like METR. METR is doing their best, but their potential future corruption—when it is revealed—would shock me not at all. "Insider" work may still be useful at this point, but anyone performing it should realize that they are in imminent peril of Hell.[1]
Atheists may substitute "utter moral corruption and betrayal of all values you once held as sacred." We're now down to gambling with a 20+% chance of human extinction (just going by what various lab leadership has said in the past), potentially in the next few years, while getting quieter and more evasive in public comms about the stakes. If that isn't a betrayal of sacred values, I don't know what is. ↩︎
I wrote a substack article about how Anthropic is not behaving sensibly according to the worldview expressed by its leaders; about how Anthropic comms is slimy; and about what it might look like if they were living up to their own ideals. It includes embedded video clips to drive the point home, and I don't know how to add those here, so I'll mostly leave this as a linkpost. Below are some excerpts around the main arguments:
[...]
[...]
[...]
[...]
[...]
[...]
Here's another convenience link to the full article, including video clips of the relevant excerpts.