If you’re already familiar with the history of the field, you might wanna skip this one…

I like to imagine future historians trying to follow the discourse around AI during the time I’ve been in the field… “Wait, so the AI ethics people think that the AI safety people are the same as the accelerationists and hate them? And the accelerationists think the safety people are the same as the ethicists and hate them? And the AI safety people want to be friends with both of them!?”

In a recent conversation with a researcher, they told me: “Yeah, I work on that, but I just do alignment, not that crazy safety stuff”. Five years ago, they might’ve said the opposite! When I wrote my PhD thesis in 2021, I said:

> Until recently, “AI safety” was the most commonly used term for technical work on reducing AI x-risk. However, “AI Alignment” is now more popular, and is more specific to x-safety (as opposed to safety more broadly).

I’m writing a pair of posts. Today’s is about “AI Safety”, tomorrow’s is about “AI Alignment”. In both cases, I’m going to walk you through the history of the term and point at the different ways people use it, and the issues I see that causing.

I’m a long-time member of (what’s currently called) “the AI (existential) safety community”, and which I’ll call “AI x-safety” to avoid confusion, i.e. the community of people concerned that AI might be an existential risk (x-risk), i.e. that it could destroy the world, or specifically, cause human extinction.

“AI safety” was adopted around 2015 in what I view as an effort to legitimize AI x-safety by broadening it to include things like “making self-driving cars not crash”, see, e.g. “Concrete Problems in AI Safety”. This seems to have been partially successful… When I joined UK AISI as a founding research director, the other research director, Yarin Gal, called himself an AI safety researcher, but not an “AI existential safety” researcher, the latter term having been popularized to fight exactly this dilution of the phrase “AI safety” that seemed strategically useful a few years prior.

Similarly, people used to talk about “short-term safety” vs. “long-term safety”. In this terminology, “AI ethics” (covering things like Fairness, Accountability, and Transparency) was lumped together with the self-driving-car-not-crash sorta stuff, despite these fields being almost entirely separate. This terminology had the problem that it baked in the assumption that existential risk was far off. This never seemed right to me, and it’s increasingly clear that Real AI may arrive within the next few years, and that it has at least some chance of causing the end of humanity.

Around the same time, others in the AI x-safety community started using the term “AI alignment” to refer specifically to the problem of “how do you get a superintelligent AI system to not go rogue and kill everyone in pursuit of “misaligned” goals (i.e. ones that are at odds with those of its creators or the interests of humanity). But now, as I mentioned at the top, we’ve come full circle, and “AI Safety” is used as short-hand for “AI x-safety”, although it also covers a larger set of concerns, as indicated by the index of the International AI Safety Report.

The main issue with the way “AI Safety” is being used these days is that it’s been conflated with wokeness and weaponized by the Trump administration and tech interests to dismiss widely held concerns around things like unemployment, killer robots, and human disempowerment and extinction. This weaponization is made easier by the fact that racial and gender bias were major targets for initial alignment/safety efforts on LLMs, although later studies showed models were actually biased against conservatives, making even the issue of “bias” a bipartisan one. I’m not sure how successful it’s been. But it did lead to some rebranding as “AI security”.

Overall, the discourse around AI Safety isn’t too poisoned by the term being used in confusing ways. But tomorrow, I’ll talk about AI Alignment, which is a whole nother can of worms…