When thinking about different techniques that aim to reduce AI risk from misaligned AIs, I find it helpful to explicitly consider multiple different levels of risk that the deployer of the AI might be going for, because different classes of safety techniques are appropriate for different risk levels. Four that I think it's particularly useful to think about:

• *The “safety case” regime.* Sometimes people talk about wanting to have approaches to safety such that if all AI developers followed these approaches, the overall level of risk posed by AI would be minimal. (These approaches are going to be more conservative than will probably be feasible in practice given the amount of competitive pressure, so I think it’s pretty likely that AI developers don’t actually hold themselves to these standards, but I agree with e.g. Anthropic that this level of caution is at least a useful hypothetical to consider.) This is the level of caution people are usually talking about when they discuss making safety cases. I usually operationalize this as the AI developer wanting to have <1% chance that their AIs escape in the first year of deployment, and <5% conditional on the model trying pretty hard to subvert their safety measures. (I think some AI developers are hoping to get lower levels of risk than this.)
• *The rushed reasonable developer regime.* The much riskier regimes I expect, where even relatively reasonable AI developers are in a huge rush and so are much less able to implement interventions carefully or to err on the side of caution.
• *The rushed unreasonable developer regime.* You’re advising an unreasonable AI developer that is incompetent at implementing safety measures and unwilling to put much effort into implementing these measures.
• *The existential war regime*. You’re in an existential war with an enemy and you’re indifferent to AI takeover vs the enemy defeating you. This might happen if you’re in a war with a nation you don’t like much, or if you’re at war with AIs. Obviously this is a regime where you should be much more aggressive.

Another option is the extreme safety regime. Sometimes people talk about approaches to AI safety that aim to ensure that an AI takeover is basically inconceivable. I think that this is highly impractical. And at the risk of saying something controversial and off-topic that I'm not confident in, I'm also not sure that it is actually a healthy way to relate to a situation as confusing as the transformation of the world from development of powerful AI. I am wary of attitudes to the world that would have led to opposing the creating of the internet, the printing press, or the industrial revolution, because I think that those things seem pretty good even though ex ante they looked pretty unpredictable. I'm in favor of trying to get particular sources of risk down to extremely low levels (e.g. I don't mind pushing to reduce asteriod risk by an OOM, and I don't mind trying to improve techniques that narrowly reduce a certain class of takeover risk), but I don't love taking this attitude to the whole AI situation. My intuition here is related to Richard Ngo's "it’s never your job to “ensure” that a large-scale risk doesn’t occur" but isn't exactly the same.