On the apocalyptic residual, or: the secret third way to save the world.

Bostrom's Vulnerable World Hypothesis says our world is vulnerable. The most commonly discussed threat model are his "Type-1 vulnerabilities": destructive technologies such as bioengineered pandemics.

There are three steps required for our world to be destroyed in this way:

1. The dangerous technology exists and is available to many people;
2. There are no defensive technologies to protect us;
3. A number of people want to destroy the world; we call those people the apocalyptic residual.

If all three are satisfied at any point in time, we die.

Most work on AI safety has been focused on (1) and (2). To be precise, here is a non-exhaustive list of efforts towards mitigating the vulnerable world hypothesis, focused on biorisk:

1. Control the availability of dangerous technology: Labs have been tracking offensive biorisk capabilities (see e.g. GDM Frontier Safety, OpenAI Preparedness, Anthropic Frontier Red Team) and developing model safeguards. There is also work on filtering pretraining data and unlearning to remove dangerous capabilities from LLMs. Regarding open-weight LLMs, there is work on biorisk capability elicitation and tamper-resistant safeguards.
2. Build defenses to dangerous technology: Vitalik's d/acc and Michael Nielsen's Notes on differential technological development have inspired a bunch of investment into defensive technology development. From the biorisk side, SecureBio exists; Open Phil has been funding work on biosecurity preparedness for a long time, and in 2025 AI labs have started funding biorisk prevention startups. To be clear, the infrastructure developed by the mRNA and Big Pharma companies since 2010 likely dwarfs the AI-lab-led and def/acc efforts in importance if we face an engineered pandemic before 2030.
3. Reduce the apocalyptic residual: ?????

I cannot think of a single person or paper working on managing the apocalyptic residual (number of people who would want to destroy the world and have enough agency to try to do so) recently.

Bostrom calls this "preference modification" and argues it is infeasible. Here are relevant quotes from the VWH paper:

It is difficult to imagine an intervention – short of radically re-engineering human nature on a fully global scale – that would sufficiently deplete the apocalyptic residual to entirely eliminate or even greatly reduce the threat of Type-1 vulnerabilities.

"Radically re-engineering human nature on a fully global scale" seems unrealistic until we realize we're about to do totally realistic things such as "automate all remote work" on the same timeframe; that technology is re-engineering human nature on a global scale; and that many people's lives are increasingly a side quest to their main task of being fed video content from variously sized screens. Status-disaffected men used to go to war or become cult leaders or something; soon they will all waste their lives on video games, their attention on short-form video feeds, their salaries on parlay bets on Robinhood or parasocial relationships with OnlyFans models; living their lives vicariously through the Content they consume and egregores that control their brain. If the forces of technocapital can coordinate to do anything, they can coordinate on preference modification of every monetizable eyeball they can reach, to make their lives as inane and inconsequential and as glued to the screen as possible.

Now, of course, the timelines of preference modification and dangerous technology might not match. Market forces might only drain the agency of a fraction of the relevant population by the critical period. But it's worth thinking about ways in which the impact of AGI actually expands society's toolset to fight x-risk.