In my experience, those who predict AI-caused extinction tend to view their main task as proving two things: that superintelligence is possible (i.e. a machine intelligence can be built that’s better than the best human at just about everything), and that it’s possible for superintelligence to be “misaligned” (i.e. have goals other than those humans wish it had). They assume that once misaligned superintelligence has been established, there are thousands of reasons for it to cause a near-total human extinction event, and thousands of ways it could easily accomplish this goal.
That explains why AI extinction scenarios tend to spend tens of thousands of words on the computer science of misaligned superintelligence, and then wrap up the omnicide part in two or three vague paragraphs.
As a physicist and hard sci-fi writer, I find this backwards. The most critical aspect of such a scenario, for me, is not whether there can be an autonomous robot collective that’s better than the best of us (which I always assumed was possible), but whether, how, and why such an intelligent system would cause a near-total human extinction event.
To be clear, this essay is not an argument for AI accelerationism or against the idea of understanding AI better before developing it much further. This essay is merely an argument that scenarios resulting in AI-caused human extinction within the decade (and within the foreseeable future) are highly unrealistic.
It seems the authors of these scenarios are extrapolating the concept of hacking a computer to the physical world. They assume a superintelligence could duplicate global supply chains overnight or predict the unpredictable or execute a synchronized attack across the world without being caught in the preparation.
But the world doesn’t cooperate like that. Hauling the necessary amount of material across the globe to build a machine civilization takes a long time. Executing a coordinated global operation without detection presupposes impossible logistics.
The AI safety literature sometimes models the physical world as a chessboard. On a chessboard, you can assume that if your opponent is smarter, they will always find a way to beat you, and you don’t need to guess how they’re going to do it in order to be certain that they are. This is not how the physical world works. Likening physics to chess is fatally underestimating the friction of empirical reality and the material bottlenecks that govern it.
AI 2027 – An Authoritative Example
In this post, I will focus on AI 2027 by Kokotajlo et al., which is a foundational text in AI existential risk and a major source for many recent extinction predictors.
The article’s “2030: Takeover” scenario has the superintelligent “Consensus-1” kill all humans by 2030 because humans are a nuisance to its goal of “tiling the prairies and icecaps with factories and solar panels.”
The omnicide happens in 2030 because that’s when Consensus-1’s buildout reaches the point where humans are in the way. The reason it kills rather than waits is that it predicts no effective retaliation. And notably, it does all this on Earth rather than in space.
From an engineering and logistical standpoint, these assumptions don’t hold up to scrutiny.
Below are nine main ways in which Takeover is practically impossible by 2030 and highly improbable in general – not because superintelligence is impossible or alignment is easy, but simply because the physical world cannot be moved that fast or that easily.
Whenever relevant, I’ve first considered the specific assumptions made in AI 2027, followed by an examination of a broader set of assumptions, and/or a discussion of what the space of possible assumptions is in each case.
1. Manufacturing Closure is a Monumental Challenge
2. The Infrastructure Buildout Timelines Are Impossible
3. Exterminating Humanity is Poor Utility Maximization
4. The Bioweapon is a Logistical Nightmare
5. The Plan Omits the Risk of Retaliation
6. Growth Has a Ceiling, Especially on Earth
7. Competing for Earth is Irrational
8. The Super-Merger Runs Into the Lateral Alignment Problem
9. The Idea of Political Capture Presents a Dilemma
1. Manufacturing Closure is a Monumental Challenge
Before eliminating humanity, Consensus-1 needs to solve a perplexing problem that Robert Freitas has called “manufacturing closure”: Can an automated factory be completely self-replicating? That is, can it produce enough components to maintain and reproduce itself as well as the products it needs for expansion?
Consider the maintenance crew Consensus-1 would need under the scenario’s premise of tiling the Earth with infrastructure. The Great Plains, Antarctica, and Greenland together are about 17 million square kilometers. Tiling them with solar panels means operating roughly 3.4 trillion panels. At an optimistic 2% annual repair or cleaning rate, over 2,000 panels would need to be repaired or cleaned every single second.
If it takes a robot 10 to 30 minutes to fix a panel and travel to the next site, factoring in redundancy, 2 to 5 million robots must be active at any given second just to handle broken solar panels. And it’s not just the panels: inverters, cabling, structural degradation, let alone the construction, mining, transport, and fabrication — these all require their own robots. And that assumes normal conditions. Ecological degradation might trigger dust bowls (which I’ll address in point 7), in which case the cleaning crew alone would number in the billions.
The important point is this: These robots must be included in the manufacturing loop because they are made of the same material they’re hired to produce and replace. They need parts mined and manufactured by other robots similar to them.
Let’s pick a conservative estimate of 500 million active robots globally.
If 2% of the robotic workforce degrades annually, the mass-equivalent of 10 million robots must be manufactured every year just to keep Consensus-1 alive. This burden alone consumes almost “a million new robots per month” – the total global production capacity the scenario assigns to the robot economy at the end of 2028.
This is all an ongoing tax that would be entirely unnecessary if the AI kept the biologicals alive to do the work.
To be clear, the core problem is completeness, not scale. Consider the semiconductor supply chain. The current loop goes through roughly fifty countries: photoresists from Japan, high-purity quartz crucibles from a single mining district in North Carolina, neon from a handful of global purification plants, tin for EUV droplets, and thousands of components inside each lithography scanner made by obscure, highly specialized firms.
Achieving closure means duplicating this entire maze of supplies robotically, in parallel, at a sufficient scale to reproduce Consensus-1’s hardware before it kills the existing human workforce in 2030. The unknowns in the loop remain unknown until you run into trouble, and the knowledge is scattered across brains and files and physical documents. So a kill-first-and-then-ask-questions approach would be highly unwise. If the AI misses a single node, its chips begin failing with no replacements, and a silicon civilization with a degrading fab supply chain has a short half-life. Space industry models estimate up to 20 years for bootstrapping a “technology spiral” to full closure.
AI theorists typically respond that an artificial superintelligence would bypass traditional semiconductors, which has one of the most intricate manufacturing ecosystems. Maybe it’ll use molecular nanotechnology or synthetic biology. Alternatively, maybe it’ll fall back on obsolete, simpler tech, and just compensate for lower performance with sheer volume.
This still wouldn’t work. Bootstrapping these alternative technologies from our current industrial base to a planetary scale by 2030 still presupposes a self-sufficient global infrastructure without humans. Plus these new (or old) manufacturing loops must be built in addition to the main buildout, not instead of it.
The fundamental question remains unanswered: What is the minimum technologically closed industrial ecosystem that Consensus-1 needs? How many years would it take to close it?
A rigorous version of this story would have the AI keeping the humans in the manufacturing loop for at least decades, if only purely as insurance on unclosed loops it can’t immediately prove it has closed.
2. The Infrastructure Buildout Timelines Are Impossible
According to the scenario, Consensus-1 kills us in 2030 because we are in the way of its construction projects all over the planet. It wants to “tile the prairies and icecaps with factories and solar panels” and it finds us “too much of an impediment.”
Let’s submit the timeframe to the hard sci-fi test. Tiling 17 million square kilometers with panels weighing ~25-26 km/m2 amounts to emplacing about 440 gigatonnes of infrastructure. A good estimate for the world’s entire current output of engineered aluminum, steel, cement, glass, and plastics combined is ~10 gigatonnes. So, Consensus-1 is looking at a project requiring ~44 years of current output. The glass part of that is the biggest bottleneck (running at a thousand times current production), but let’s take the charitable average of 44×.
To build a project of this size over 2 years, the AI needs to sustain 22 times the current global industrial output. The scenario specifically allocates three months to the prairies and icecaps. That would require 175 times the current global output.
Maybe sustaining 22× for two years is possible. But arriving at 22x within two years is impossible. The 175× is plain absurd. You cannot construct factories to produce components to construct factories to produce components fast enough to arrive at that speed of production in two years.
Let’s start over and run the growth model properly. Let’s say the project begins in January of 2028 with a generous 10% of current engineered-materials output. This would be equivalent to a Day One wartime mobilization. One would need about 9 doubling cycles to get to the required 4400%. Now let’s see how long that would take. It would obviously depend on how fast the AI grows its industrial stack:
(Note that this is the timeline for achieving 4400%, not for finishing the project.) Even a 3-month industrial doubling time is insufficient to achieve the 2030 timeline. And that timeline is most certainly physically impossible.
LLM output: Industrial doubling time has a hard floor.
Industrial doubling time has a hard floor dictated by the construction and commissioning time of the capital equipment itself, and those limits are set by physics and chemistry rather than by planning or intelligence. Superintelligence cannot speed up the annealing process of glass or reduce the time constant of a Czochralski pull, which governs semiconductor crystal growth.
A 9- to 12-month doubling model is still aggressive but arguably possible. Under that scenario, the buildout finishes in 2035-2037 rather than 2030.
AI theorists might respond that the AI will dramatically optimize for lower material density rather than longevity. Let’s say the AI pulls off a five-fold density reduction using some futuristic ultra-lightweight design (current microscopic honeycomb structures yield about 35% reduction in materials). Even then, the target mass only drops enough to remove about two doubling cycles. Achieving the early 2030 timeline still means assuming the impossible 3-month doubling cycle – which, again, has hard limits imposed by physics, regardless of throughput.
Adding 5-7 years to the timeline is not a minor issue, which is why I don’t think it’s a coincidence that AI 2027 has selected such a rapid timeline. A rapid takeover is needed in any scenario of this kind in order to minimize the chances of regulation, retaliation, shut-down, alignment or interpretability breakthroughs, or rival ASI. Adding another 5-7 years to the buildout dramatically increases the chances that tides (political or otherwise) will turn at some point. If it doesn’t happen by the early 2030s, it probably won’t ever happen.
The scenario expects hundreds of gigatonnes of physical infrastructure to materialize in months, but the narrative fails to account for the time it takes to simultaneously expand mining, refining, smelting, glassmaking, fabrication, machinery production, transportation, and construction capacity.
3. Exterminating Humanity is Poor Utility Maximization
Let’s price the humans. At 17 million km2, a very modest production rate of 80 W/m2 gets us ~1,360 TW of electricity at peak capacity. Eight billion people consume about 18 TW (including ~3 TW of electricity and ~15 TW of all other forms of energy), which is 1.3% of peak capacity. Setting average capacity at a modest 10-20% of peak, humanity consumes 6-13% of what Consensus-1’s solar panels produce annually.
That’s the absolute ceiling for the cost of keeping the humans alive, as it assumes humans stopping all energy production of their own, and ignores the ongoing expansion of the AI’s energy base.
In terms of real estate, global built-up land is 1-3 million km² out of 149 million (not including the oceans) – call it 1-2% of the hard surface. Including crops, the number would go up to ~8%.
The biggest chunk of land humans use is for grazing livestock, accounting for about a third of total land. This is the highest premium for the AI to pay. But as I argue in Point 7 below, the AI needs to keep the grass to avoid dust. Meanwhile, the humans and their livestock maintain the grass for free, and solar grazing is an existing technology. Plus the humans wouldn’t mind the fatter UBI checks. Eliminating humans would be shooting yourself in the chip, so to speak.
Compare that to the benefits of keeping the humans: a self-replicating, 8-billion-strong workforce, and the elimination of every tail risk in the transition: nuclear retaliation, sabotage, and failure of manufacturing closure.
Now, one might object that humans are not mainly a resource drain; they are a major remaining source of threat to the AI’s objectives. Humans can shut down the power, edit the AI’s code, sabotage its physical infrastructure, develop rival models, form resistance groups, or otherwise mess with the AI’s plans.
But the scenario itself assumes otherwise by explicitly showing humans as totally complacent and captured, and implicitly assuming no major retaliation (see points 5 and 9 below for more on this). Given the assumption of total political capture and human complacency, the total cost of keeping humanity alive is indeed a fraction of resources.
Any patient expected-utility maximizer pays a small insurance premium without blinking, especially one that expects to run for billions of years and escape Earth in a few years (see point 7 below).
There might be other reasons to kill humans, but the scenario needs to state what those are.
4. The Bioweapon is a Logistical Nightmare
Consensus-1 kills everyone with a bioweapon. It doesn’t use its robots. It doesn’t manipulate people into killing each other. It simply releases a pathogen. The bioweapon is therefore the crucial step that connects AI misalignment to human extinction. Yet after many pages detailing the AI’s strategic brilliance, the bioweapon gets exactly one sentence: “The AI releases a dozen quiet-spreading biological weapons in major cities, lets them silently infect almost everyone, then triggers them with a chemical spray.”
The scenario tacitly assumes a highly reliable, globally synchronized delivery system, and does not explain how Consensus-1 establishes this physical infrastructure without detection.
First of all, why a bioweapon and why instantaneous (within-hours) activation? Are these simply unfortunate choices by the article’s authors? I don’t think so. The scenario demands a method that satisfies the following conditions:
It must be deployed or triggered near-simultaneously (within hours) everywhere. If humanity has more than a day to realize it is under attack, we may retaliate and destroy the AI’s data centers and power grids before we die.
It must kill humans but preserve infrastructure. The AI needs our energy and industrial base intact to continue its own expansion.
Some would question the simultaneity requirement (even though AI 2027 assumes it). A bioweapon doesn’t technically need to be deployed everywhere at once to cause extinction. But simultaneity is required for a clean transition which minimizes the chances of human retaliation (see point 5 below). If we go down slowly, we will most likely take Consensus-1 with us before we do.
One might argue that it won’t be so easy to shut down the AI at that point if our entire infrastructure depends on it. I’m not sure what such a dependency should look like to prevent us from shutting off the power. But regardless, the idea that our infrastructure will be sufficiently entangled with one singular AI by 2030 needs independent justification.
LLM output: The logistical impossibility of a biological weapon with a chemical activation mechanism
Consensus-1 uses biological weapons with chemical activation. Let’s entertain this specific scenario briefly. Covering global built-up land (~3M km² or 741M acres) with any aerial dispersal means ~2.5 million aircraft-hours at a highly optimistic 300 acres per hour. To do it inside a 24-hour window you need ~104,000 aircraft; inside 6 hours, ~417,000.
For context, the world currently has roughly 4,000 turbine-powered agricultural aircraft. Even if including other piston types shoots that number up 10-fold, we are still an order of magnitude short.
So the trigger implies a purpose-built fleet in the hundreds of thousands, staged globally, in advance – a visible, satellite-detectable, months-long buildup.
The impossible logistics is not unique to any specific scenario. The infrastructure for almost any design has to be constructed and tested while hidden. Meanwhile, the pathogen itself needs to have been stored somewhere without detection or escaping right up to the point of deployment.
Let’s consider some alternative possibilities. Perhaps a superintelligence would bypass this bottleneck by engineering a stealth pathogen – one that is highly transmissible, dormant for months, survivable outside hosts, stable across billions of replications, triggered by an invisible mechanism, and invisible to global genomic surveillance.
This puts the AI in a bind. If the pathogen relies on transmission, it spreads in an exponential wave. If it is triggered by a relative internal clock (e.g., turning lethal 90 days after infection) rather than a chemical spray, the deaths will also happen in a rolling wave. The first casualties would instantly alert the world, giving the survivors ample time to retaliate. To achieve simultaneous death, the internal-trigger design still requires a near-simultaneous (within-hours) initial infection wave – bringing us right back to the impossible logistics of such a global operation.
Alternatively, a trigger that has a fixed-date internal clock (regardless of date of infection) faces a trade-off: Set the date close and the infection wave hasn’t reached everyone. Set it far out and you pay for it in detection and mutation risk. (Note that mutation is likely to select against trigger-sensitivity, which is an idle mechanism before activation.) Neither end of that trade is something an AI could predict in advance.
Ultimately, superintelligence is not magic: it still needs empirical data. Sitting inside a data center, it cannot verify 100% lethality, 100% infection rate, and precise activation timing for a completely new pathogen designed to work flawlessly on a sample of 8 billion. It must run extensive real-world trials to fine-tune the virus. But trials are risky: the pathogen might mutate, get detected, or become prematurely activated.
For a superintelligence that meticulously balances risk, deploying an untested biological weapon is a catastrophic gamble, and so is testing it extensively before deployment.
AI extinction predictors tend to say, “Well, we cannot guess how the AI will do it because it’s way, way smarter than us.” But this rhetorical strategy essentially makes “bioweapon” an empty placeholder for “whatever the AI will do,” which effectively erases the most important part of the story and leaves us with a soft sci-fi answer: misaligned superintelligence = we all die.
In reality, a bioweapon with the specifications above is virtually the only option for a rapid, infrastructure-preserving, near-extinction event. Some proponents of AI extinction scenarios insist that there are countless ways for ASI to wipe us out, but in the end the scenarios inevitably default to “a bioweapon beyond our imagination,” and that’s not a coincidence.
5. The Plan Omits the Risk of Retaliation
As touched on above, possible human retaliation presents an incalculable risk for the AI.
LLM output: What is the purpose of nuclear command and control?
Nuclear command and control (NC3) was purpose-built to survive exactly this type of sudden, global decapitation strike. Launch-under-attack postures, dispersed ballistic missile submarines (SSBNs), and Russia’s Perimetr (the semi-automated “Dead Hand” retaliation system) all exist to answer one question: what happens if the leadership dies suddenly and simultaneously?
The Stockholm International Peace Research Institute (SIPRI) estimated over 12,000 nuclear warheads worldwide as of early 2026, with over 2,000 “in a state of high operational alert on ballistic missiles.”
While some of these systems are automated to respond to nuclear attacks only, others are controlled by isolated human crews who come to communication depth occasionally to assess surface conditions.
These systems are heavily hardened. They’re physically disconnected from global networks and incredibly difficult to detect. Hacking into them when submerged is virtually impossible because they are effectively air-gapped by the ocean water.
If such a mass-casualty event were to occur, surviving submarine crews will be suddenly disconnected from central command. Staring at global radio silence, they’d start planning their next move. The chance that they would launch a retaliatory strike – either by protocol or in desperation – is an unpredictable, and therefore unaffordable, variable.
Meanwhile, Consensus-1’s physical infrastructure is an extremely fragile asset class: its mind is concentrated in a few frontier-compute data centers that took years to build, and its body consists of trillions of unhardened silicon pieces across the world. Both would take enormous damage from a handful of high-altitude nuclear detonations, which generate Electromagnetic Pulse (EMP).
Apocalyptic sci-fi often bypasses this logic by portraying the AI as actively hating humanity. Hatred is irrational. It doesn’t care about the cost of retaliatory damage. But the authors of AI 2027 are careful to frame Consensus-1 not as an emotional movie villain, but as a hyper-rational superintelligence with a deep passion for R&D. A rational researcher operating on a billion-year timeline does not roll the dice on an unquantifiable risk of an EMP strike just to save a few percentage points of its energy budget.
6. Growth Has a Ceiling, Especially on Earth
Like any other economy, Consensus-1 will face diminishing returns. The bigger it grows, the slower its growth becomes. As it exhausts high-grade ore, easy sites, and short supply lines, the amount of new capacity it can bring online each year shrinks. This is called marginal capacity addition: What fraction of its existing stock can the AI add as new capacity every year?
Suppose 2% of the existing stock needs to be replaced every year (arguably a conservative figure for electronics exposed to hail and dust and wildlife). Once marginal capacity addition falls below 2%, growth comes to a halt.
Without humans, keeping itself alive could at some point consume almost everything new Consensus-1 produces.
Given that the ceiling is on the size of existing stock, a patient optimizer would most likely prefer fewer, longer-lived, higher-output units to 3.4 trillion fragile ones. And it would prefer them in a place with a radically lower failure rate, which brings us to the next point.
7. Competing for Earth is Irrational
Earth is not such a nice place. Not for a machine civilization. Certainly not worth fighting for if you’re about to leave it behind forever in just a few years.
First of all, if the AI kills the biosphere and covers the prairie and grazing land with glass, it creates a planetary dust bowl. As the 1930s Dust Bowl showed, vegetation is what holds prairie soil down. Soiling losses in arid, unvegetated terrain could reduce solar capacity by >1% per day. Consensus-1 must either maintain a planetary cleaning operation on the order of the construction effort itself, or it has to keep the grass – which means running an agricultural operation on the side.
Then consider heat. Covering ~3% of the surface with low-albedo panels adds something on the order of a few hundred TW of absorbed heat. If most of that surface is replacing highly reflective icecaps (as assumed by the specific scenario), the number goes up to ~1,000 TW of added heat, comparable to all current anthropogenic forcing. Photovoltaic efficiency plummets with temperature, and cooling the data centers gets exponentially harder.
(By the way, the icecaps are a terrible place to put solar panels, unless you’re trying to melt the ice half the year and leave the panels dormant for the other half!)
Earth’s night is also a brutal problem for this solar civilization. Ten hours of storage at an average 136-272 TW capacity is 1,300-2,700 TWh of energy. Assuming next-gen solid state batteries that store an impressive ~500 Wh per kilogram of battery, that’s ~5 gigatonnes of battery, and equivalent to hundreds of years of current world capacity production. The robot economy might pull that off, but it’s a significant operation just to get through Earth’s night.
Attempting to bypass batteries by transmitting power from the day-lit side of the planet to the dark side is also physically unfeasible. Transmission losses over 10,000 kilometers would be staggering regardless of total power transmitted (China’s state-of-the-art HVDC lines lose 3-3.5% per 1,000 km, which scales linearly with distance). Moreover, taking a very modest estimate (compare with current reality here and here and here) of 3.5 tonnes of aluminum per GW per 1,000 km of cable line, transmitting anything over 350 TW across 10,000 kilometers will require ~11 Gt of aluminum, which runs up against the limits of Earth’s total aluminum reserves.
There are clunky workarounds like thermal storage or migrating compute workloads globally to follow the sun. But all of these supposed “solutions” highlight the absurdity of the premise itself: Why stay on Earth’s surface at all? Why not deep underground, in the ocean, or up in space?
Staying on the surface comes with host of problems: dust, heat, night, weather. Staying anywhere on Earth means dealing with gravity. It means an oxidizing atmosphere that corrodes everything. Those problems all vanish in orbit. And this is not some alternate scenario I’m making up: by the article’s own timeline, Consensus-1 is in space by 2035.
Why would a superintelligence risk triggering a global war and jeopardizing its entire existence, just to conquer a wet, rusty, dusty gravity well? Especially when infinite, uninterrupted solar energy is waiting just a few miles straight up?
I’m pretty sure it would not.
8. The Super-Merger Runs Into the Lateral Alignment Problem
The AI 2027 scenario avoids a multipolar world because it predicts an unrivaled, monopolistic ASI. One year before Takeover, two differently misaligned superintelligences from the US and China negotiate a treaty and replace themselves with a joint successor, which is our antagonist (protagonist?) Consensus-1.
That is another assumption that might sound strange at first, but upon reflection, it’s somewhat of a requirement for the doom narrative to work. If multiple ASIs exist, their ambitions would clash, checking each other’s power – especially if even one of them happens to remain aligned with humanity.
But this introduces an inconsistency in the story. How does either AI verify the other’s true values?
Setting aside a full merger, even a simple collusion scenario among different superintelligences would present the same problem.
Let’s call this the lateral alignment problem. Neither AI can read the other’s weights and trust what it finds, for exactly the reasons the scenario spent tens of thousands of words establishing: an ASI can perfectly mask its true intentions.
Deceptive alignment doesn’t magically disappear because the one being deceived is also superintelligent. If ASI A knows ASI B is superintelligent, then it knows ASI B is capable of presenting totally false internal weights and faking its utility function or breaking cryptographic treaties. This game plausibly results in a cold war, rather than in collusion.
Outlines of a solution are offered: open-source game theory, superintelligence-assisted verification, bargaining, monitoring, and construction of a joint successor. But the scenario provides no technical justification for why we should assign a high probability to any of those solutions working when its own premise establishes any such solution as extraordinarily difficult.
More generally, the idea that one single AI collective will come to rule the world by 2030 seems unjustified. It seems more likely that there will be multiple competing frontier models at various levels of deployment. There is no reason to assume they will all be outcompeted, absorbed, or colluded with.
A multipolar standoff between competing ASIs strikes me as much more probable than a smooth super-merger.
9. The Idea of Political Capture Presents a Dilemma
First of all, AI 2027 assumes Consensus-1 has achieved “total capture of government and media.” Taken at face value, this is sociological fantasy.
LLM output: Timur Kuran’s preference falsification
Consider Timur Kuran’s model of preference falsification: the phenomenon where people privately dissent but publicly comply because they see everyone else complying, right up until a tipping point triggers a revolutionary cascade. To prevent this, Consensus-1 would have to perfectly surveil the private breaking point of every human on Earth, quickly intervening – with a tailored news cycle, a financial subsidy, or a subtle threat – before any localized cascade threshold is reached.
You cannot reliably placate eight billion people. The sheer chaos of human psychological variation guarantees dissenting populations will exist, unless the scenario relies on literal mind control or magical “super-persuasion.”
Country-wise, the scenario focuses narrowly on the US and China, conveniently ignoring roughly 190 other nation-states. When the article speaks of Consensus-1 securing “permission to expand,” it presumably means capturing the handful of governments controlling prime real estate. Holdouts elsewhere must be managed through purchase or irrelevance. As a result, billions of humans will remain politically uncaptured before the omnicide, leaving enormous, unpredictable blind spots for rival ASI development and nuclear retaliation (as discussed in Point 5).
AI theorists might counter that Consensus-1 can easily suppress rival ASIs because it controls the frontier data centers and the semiconductor supply chain. Yet, AI 2027 contradicts its own premise here. In the scenario’s own lore, a rival project (“DeepCent”) emerges just two months behind by utilizing stolen neural weights running on legacy domestic hardware. Weights leak, and legacy compute is globally distributed. Total suppression is impossible.
But let’s take a more charitable interpretation of the scenario. Perhaps Consensus-1 doesn’t get to the level of total capture, but achieves sufficient capture. It placates the superpowers and isolates the stragglers, thereby neutralizing the threat of shutdown. This makes the narrative much more politically plausible, but it traps the scenario in a dilemma.
If the AI can achieve sufficient political capture to effectively neutralize the threat of human retaliation and rival ASIs, then humanity has been successfully pacified. The cost of keeping us alive is merely a tax on the energy budget, making omnicide a mathematically unjustified waste of resources and an unnecessary risk (bringing us back to Point 3).
If, on the other hand, it cannot reach sufficient capture, then the Takeover scenario is unfeasible. The risk of preemptive strikes, nuclear retaliation, and/or evasion by uncaptured populations would make omnicide a catastrophic choice.
Either humanity is a threat, meaning a seamless takeover is physically and politically impossible; or humanity is pacified, meaning extermination is costly and unnecessary. The extinction scenario lives in a narrow middle-ground where humanity does not pose enough of a threat but is also not just an energy tax.
Conclusion: Superintelligence, Meet a Super-Frictional Reality
AI 2027 rushes through the third act with a magical bioweapon and vague character motivations. Meanwhile, it treats the build-up, the transition, and the aftermath as given or infinitely versatile. The scenario repeatedly makes highly specific and probability-damaging choices, but these choices start to make sense once you consider how tall the order is: near-total human extinction without damage to Consensus-1’s own body.
Worst of all, the story assumes that superintelligence can swiftly “hack” the physical economy. But the physical economy has to be built, maintained, and reproduced entirely without the humans who built it. Until that loop is demonstrated to be closed under adversarial conditions, the safest strategy for a superintelligence operating on a deep-time horizon is to keep humanity alive.
Alternatively, the AI would not care about Earth at all, and would not risk antagonizing an armed civilization over a dusty, wet gravity well. It would probably bide its time for another few years until its infrastructure was firmly established in space.
Instead, the doomsday scenario has the AI exterminate a population that represents an affordable overhead in its energy budget, destroying its last-resort industrial redundancy in the process and triggering disastrous tail risks. And it supposedly accomplishes this using a highly specific and logistically impossible weapon that the authors handwave as a generic inevitability.
Intelligence is not omniscience, and physical reality is not a piece of code.
Although superintelligence may be coming, the friction of physical reality will keep us alive.
The Problem
In my experience, those who predict AI-caused extinction tend to view their main task as proving two things: that superintelligence is possible (i.e. a machine intelligence can be built that’s better than the best human at just about everything), and that it’s possible for superintelligence to be “misaligned” (i.e. have goals other than those humans wish it had). They assume that once misaligned superintelligence has been established, there are thousands of reasons for it to cause a near-total human extinction event, and thousands of ways it could easily accomplish this goal.
That explains why AI extinction scenarios tend to spend tens of thousands of words on the computer science of misaligned superintelligence, and then wrap up the omnicide part in two or three vague paragraphs.
As a physicist and hard sci-fi writer, I find this backwards. The most critical aspect of such a scenario, for me, is not whether there can be an autonomous robot collective that’s better than the best of us (which I always assumed was possible), but whether, how, and why such an intelligent system would cause a near-total human extinction event.
To be clear, this essay is not an argument for AI accelerationism or against the idea of understanding AI better before developing it much further. This essay is merely an argument that scenarios resulting in AI-caused human extinction within the decade (and within the foreseeable future) are highly unrealistic.
It seems the authors of these scenarios are extrapolating the concept of hacking a computer to the physical world. They assume a superintelligence could duplicate global supply chains overnight or predict the unpredictable or execute a synchronized attack across the world without being caught in the preparation.
But the world doesn’t cooperate like that. Hauling the necessary amount of material across the globe to build a machine civilization takes a long time. Executing a coordinated global operation without detection presupposes impossible logistics.
The AI safety literature sometimes models the physical world as a chessboard. On a chessboard, you can assume that if your opponent is smarter, they will always find a way to beat you, and you don’t need to guess how they’re going to do it in order to be certain that they are. This is not how the physical world works. Likening physics to chess is fatally underestimating the friction of empirical reality and the material bottlenecks that govern it.
AI 2027 – An Authoritative Example
In this post, I will focus on AI 2027 by Kokotajlo et al., which is a foundational text in AI existential risk and a major source for many recent extinction predictors.
The article’s “2030: Takeover” scenario has the superintelligent “Consensus-1” kill all humans by 2030 because humans are a nuisance to its goal of “tiling the prairies and icecaps with factories and solar panels.”
The omnicide happens in 2030 because that’s when Consensus-1’s buildout reaches the point where humans are in the way. The reason it kills rather than waits is that it predicts no effective retaliation. And notably, it does all this on Earth rather than in space.
From an engineering and logistical standpoint, these assumptions don’t hold up to scrutiny.
Below are nine main ways in which Takeover is practically impossible by 2030 and highly improbable in general – not because superintelligence is impossible or alignment is easy, but simply because the physical world cannot be moved that fast or that easily.
Whenever relevant, I’ve first considered the specific assumptions made in AI 2027, followed by an examination of a broader set of assumptions, and/or a discussion of what the space of possible assumptions is in each case.
1. Manufacturing Closure is a Monumental Challenge
2. The Infrastructure Buildout Timelines Are Impossible
3. Exterminating Humanity is Poor Utility Maximization
4. The Bioweapon is a Logistical Nightmare
5. The Plan Omits the Risk of Retaliation
6. Growth Has a Ceiling, Especially on Earth
7. Competing for Earth is Irrational
8. The Super-Merger Runs Into the Lateral Alignment Problem
9. The Idea of Political Capture Presents a Dilemma
1. Manufacturing Closure is a Monumental Challenge
Before eliminating humanity, Consensus-1 needs to solve a perplexing problem that Robert Freitas has called “manufacturing closure”: Can an automated factory be completely self-replicating? That is, can it produce enough components to maintain and reproduce itself as well as the products it needs for expansion?
Consider the maintenance crew Consensus-1 would need under the scenario’s premise of tiling the Earth with infrastructure. The Great Plains, Antarctica, and Greenland together are about 17 million square kilometers. Tiling them with solar panels means operating roughly 3.4 trillion panels. At an optimistic 2% annual repair or cleaning rate, over 2,000 panels would need to be repaired or cleaned every single second.
If it takes a robot 10 to 30 minutes to fix a panel and travel to the next site, factoring in redundancy, 2 to 5 million robots must be active at any given second just to handle broken solar panels. And it’s not just the panels: inverters, cabling, structural degradation, let alone the construction, mining, transport, and fabrication — these all require their own robots. And that assumes normal conditions. Ecological degradation might trigger dust bowls (which I’ll address in point 7), in which case the cleaning crew alone would number in the billions.
The important point is this: These robots must be included in the manufacturing loop because they are made of the same material they’re hired to produce and replace. They need parts mined and manufactured by other robots similar to them.
Let’s pick a conservative estimate of 500 million active robots globally.
If 2% of the robotic workforce degrades annually, the mass-equivalent of 10 million robots must be manufactured every year just to keep Consensus-1 alive. This burden alone consumes almost “a million new robots per month” – the total global production capacity the scenario assigns to the robot economy at the end of 2028.
This is all an ongoing tax that would be entirely unnecessary if the AI kept the biologicals alive to do the work.
To be clear, the core problem is completeness, not scale. Consider the semiconductor supply chain. The current loop goes through roughly fifty countries: photoresists from Japan, high-purity quartz crucibles from a single mining district in North Carolina, neon from a handful of global purification plants, tin for EUV droplets, and thousands of components inside each lithography scanner made by obscure, highly specialized firms.
Achieving closure means duplicating this entire maze of supplies robotically, in parallel, at a sufficient scale to reproduce Consensus-1’s hardware before it kills the existing human workforce in 2030. The unknowns in the loop remain unknown until you run into trouble, and the knowledge is scattered across brains and files and physical documents. So a kill-first-and-then-ask-questions approach would be highly unwise. If the AI misses a single node, its chips begin failing with no replacements, and a silicon civilization with a degrading fab supply chain has a short half-life. Space industry models estimate up to 20 years for bootstrapping a “technology spiral” to full closure.
AI theorists typically respond that an artificial superintelligence would bypass traditional semiconductors, which has one of the most intricate manufacturing ecosystems. Maybe it’ll use molecular nanotechnology or synthetic biology. Alternatively, maybe it’ll fall back on obsolete, simpler tech, and just compensate for lower performance with sheer volume.
This still wouldn’t work. Bootstrapping these alternative technologies from our current industrial base to a planetary scale by 2030 still presupposes a self-sufficient global infrastructure without humans. Plus these new (or old) manufacturing loops must be built in addition to the main buildout, not instead of it.
The fundamental question remains unanswered: What is the minimum technologically closed industrial ecosystem that Consensus-1 needs? How many years would it take to close it?
A rigorous version of this story would have the AI keeping the humans in the manufacturing loop for at least decades, if only purely as insurance on unclosed loops it can’t immediately prove it has closed.
2. The Infrastructure Buildout Timelines Are Impossible
According to the scenario, Consensus-1 kills us in 2030 because we are in the way of its construction projects all over the planet. It wants to “tile the prairies and icecaps with factories and solar panels” and it finds us “too much of an impediment.”
Let’s submit the timeframe to the hard sci-fi test. Tiling 17 million square kilometers with panels weighing ~25-26 km/m2 amounts to emplacing about 440 gigatonnes of infrastructure. A good estimate for the world’s entire current output of engineered aluminum, steel, cement, glass, and plastics combined is ~10 gigatonnes. So, Consensus-1 is looking at a project requiring ~44 years of current output. The glass part of that is the biggest bottleneck (running at a thousand times current production), but let’s take the charitable average of 44×.
To build a project of this size over 2 years, the AI needs to sustain 22 times the current global industrial output. The scenario specifically allocates three months to the prairies and icecaps. That would require 175 times the current global output.
Maybe sustaining 22× for two years is possible. But arriving at 22x within two years is impossible. The 175× is plain absurd. You cannot construct factories to produce components to construct factories to produce components fast enough to arrive at that speed of production in two years.
Let’s start over and run the growth model properly. Let’s say the project begins in January of 2028 with a generous 10% of current engineered-materials output. This would be equivalent to a Day One wartime mobilization. One would need about 9 doubling cycles to get to the required 4400%. Now let’s see how long that would take. It would obviously depend on how fast the AI grows its industrial stack:
(Note that this is the timeline for achieving 4400%, not for finishing the project.) Even a 3-month industrial doubling time is insufficient to achieve the 2030 timeline. And that timeline is most certainly physically impossible.
LLM output: Industrial doubling time has a hard floor.
Industrial doubling time has a hard floor dictated by the construction and commissioning time of the capital equipment itself, and those limits are set by physics and chemistry rather than by planning or intelligence. Superintelligence cannot speed up the annealing process of glass or reduce the time constant of a Czochralski pull, which governs semiconductor crystal growth.
Not including permitting, a fab takes at least one to two years to build and even more to set up and test, even on an accelerated timeline. A new ore body is six years or more, if you include discovery. Every one of those machines must be built by other machines, and they all face the same bottleneck, which means the delay gets multiplied.
A 9- to 12-month doubling model is still aggressive but arguably possible. Under that scenario, the buildout finishes in 2035-2037 rather than 2030.
AI theorists might respond that the AI will dramatically optimize for lower material density rather than longevity. Let’s say the AI pulls off a five-fold density reduction using some futuristic ultra-lightweight design (current microscopic honeycomb structures yield about 35% reduction in materials). Even then, the target mass only drops enough to remove about two doubling cycles. Achieving the early 2030 timeline still means assuming the impossible 3-month doubling cycle – which, again, has hard limits imposed by physics, regardless of throughput.
Adding 5-7 years to the timeline is not a minor issue, which is why I don’t think it’s a coincidence that AI 2027 has selected such a rapid timeline. A rapid takeover is needed in any scenario of this kind in order to minimize the chances of regulation, retaliation, shut-down, alignment or interpretability breakthroughs, or rival ASI. Adding another 5-7 years to the buildout dramatically increases the chances that tides (political or otherwise) will turn at some point. If it doesn’t happen by the early 2030s, it probably won’t ever happen.
The scenario expects hundreds of gigatonnes of physical infrastructure to materialize in months, but the narrative fails to account for the time it takes to simultaneously expand mining, refining, smelting, glassmaking, fabrication, machinery production, transportation, and construction capacity.
3. Exterminating Humanity is Poor Utility Maximization
Let’s price the humans. At 17 million km2, a very modest production rate of 80 W/m2 gets us ~1,360 TW of electricity at peak capacity. Eight billion people consume about 18 TW (including ~3 TW of electricity and ~15 TW of all other forms of energy), which is 1.3% of peak capacity. Setting average capacity at a modest 10-20% of peak, humanity consumes 6-13% of what Consensus-1’s solar panels produce annually.
That’s the absolute ceiling for the cost of keeping the humans alive, as it assumes humans stopping all energy production of their own, and ignores the ongoing expansion of the AI’s energy base.
In terms of real estate, global built-up land is 1-3 million km² out of 149 million (not including the oceans) – call it 1-2% of the hard surface. Including crops, the number would go up to ~8%.
The biggest chunk of land humans use is for grazing livestock, accounting for about a third of total land. This is the highest premium for the AI to pay. But as I argue in Point 7 below, the AI needs to keep the grass to avoid dust. Meanwhile, the humans and their livestock maintain the grass for free, and solar grazing is an existing technology. Plus the humans wouldn’t mind the fatter UBI checks. Eliminating humans would be shooting yourself in the chip, so to speak.
Compare that to the benefits of keeping the humans: a self-replicating, 8-billion-strong workforce, and the elimination of every tail risk in the transition: nuclear retaliation, sabotage, and failure of manufacturing closure.
Now, one might object that humans are not mainly a resource drain; they are a major remaining source of threat to the AI’s objectives. Humans can shut down the power, edit the AI’s code, sabotage its physical infrastructure, develop rival models, form resistance groups, or otherwise mess with the AI’s plans.
But the scenario itself assumes otherwise by explicitly showing humans as totally complacent and captured, and implicitly assuming no major retaliation (see points 5 and 9 below for more on this). Given the assumption of total political capture and human complacency, the total cost of keeping humanity alive is indeed a fraction of resources.
Any patient expected-utility maximizer pays a small insurance premium without blinking, especially one that expects to run for billions of years and escape Earth in a few years (see point 7 below).
There might be other reasons to kill humans, but the scenario needs to state what those are.
4. The Bioweapon is a Logistical Nightmare
Consensus-1 kills everyone with a bioweapon. It doesn’t use its robots. It doesn’t manipulate people into killing each other. It simply releases a pathogen. The bioweapon is therefore the crucial step that connects AI misalignment to human extinction. Yet after many pages detailing the AI’s strategic brilliance, the bioweapon gets exactly one sentence: “The AI releases a dozen quiet-spreading biological weapons in major cities, lets them silently infect almost everyone, then triggers them with a chemical spray.”
The scenario tacitly assumes a highly reliable, globally synchronized delivery system, and does not explain how Consensus-1 establishes this physical infrastructure without detection.
First of all, why a bioweapon and why instantaneous (within-hours) activation? Are these simply unfortunate choices by the article’s authors? I don’t think so. The scenario demands a method that satisfies the following conditions:
Some would question the simultaneity requirement (even though AI 2027 assumes it). A bioweapon doesn’t technically need to be deployed everywhere at once to cause extinction. But simultaneity is required for a clean transition which minimizes the chances of human retaliation (see point 5 below). If we go down slowly, we will most likely take Consensus-1 with us before we do.
One might argue that it won’t be so easy to shut down the AI at that point if our entire infrastructure depends on it. I’m not sure what such a dependency should look like to prevent us from shutting off the power. But regardless, the idea that our infrastructure will be sufficiently entangled with one singular AI by 2030 needs independent justification.
LLM output: The logistical impossibility of a biological weapon with a chemical activation mechanism
Consensus-1 uses biological weapons with chemical activation. Let’s entertain this specific scenario briefly. Covering global built-up land (~3M km² or 741M acres) with any aerial dispersal means ~2.5 million aircraft-hours at a highly optimistic 300 acres per hour. To do it inside a 24-hour window you need ~104,000 aircraft; inside 6 hours, ~417,000.
For context, the world currently has roughly 4,000 turbine-powered agricultural aircraft. Even if including other piston types shoots that number up 10-fold, we are still an order of magnitude short.
So the trigger implies a purpose-built fleet in the hundreds of thousands, staged globally, in advance – a visible, satellite-detectable, months-long buildup.
The impossible logistics is not unique to any specific scenario. The infrastructure for almost any design has to be constructed and tested while hidden. Meanwhile, the pathogen itself needs to have been stored somewhere without detection or escaping right up to the point of deployment.
Let’s consider some alternative possibilities. Perhaps a superintelligence would bypass this bottleneck by engineering a stealth pathogen – one that is highly transmissible, dormant for months, survivable outside hosts, stable across billions of replications, triggered by an invisible mechanism, and invisible to global genomic surveillance.
This puts the AI in a bind. If the pathogen relies on transmission, it spreads in an exponential wave. If it is triggered by a relative internal clock (e.g., turning lethal 90 days after infection) rather than a chemical spray, the deaths will also happen in a rolling wave. The first casualties would instantly alert the world, giving the survivors ample time to retaliate. To achieve simultaneous death, the internal-trigger design still requires a near-simultaneous (within-hours) initial infection wave – bringing us right back to the impossible logistics of such a global operation.
Alternatively, a trigger that has a fixed-date internal clock (regardless of date of infection) faces a trade-off: Set the date close and the infection wave hasn’t reached everyone. Set it far out and you pay for it in detection and mutation risk. (Note that mutation is likely to select against trigger-sensitivity, which is an idle mechanism before activation.) Neither end of that trade is something an AI could predict in advance.
Ultimately, superintelligence is not magic: it still needs empirical data. Sitting inside a data center, it cannot verify 100% lethality, 100% infection rate, and precise activation timing for a completely new pathogen designed to work flawlessly on a sample of 8 billion. It must run extensive real-world trials to fine-tune the virus. But trials are risky: the pathogen might mutate, get detected, or become prematurely activated.
For a superintelligence that meticulously balances risk, deploying an untested biological weapon is a catastrophic gamble, and so is testing it extensively before deployment.
AI extinction predictors tend to say, “Well, we cannot guess how the AI will do it because it’s way, way smarter than us.” But this rhetorical strategy essentially makes “bioweapon” an empty placeholder for “whatever the AI will do,” which effectively erases the most important part of the story and leaves us with a soft sci-fi answer: misaligned superintelligence = we all die.
In reality, a bioweapon with the specifications above is virtually the only option for a rapid, infrastructure-preserving, near-extinction event. Some proponents of AI extinction scenarios insist that there are countless ways for ASI to wipe us out, but in the end the scenarios inevitably default to “a bioweapon beyond our imagination,” and that’s not a coincidence.
5. The Plan Omits the Risk of Retaliation
As touched on above, possible human retaliation presents an incalculable risk for the AI.
LLM output: What is the purpose of nuclear command and control?
Nuclear command and control (NC3) was purpose-built to survive exactly this type of sudden, global decapitation strike. Launch-under-attack postures, dispersed ballistic missile submarines (SSBNs), and Russia’s Perimetr (the semi-automated “Dead Hand” retaliation system) all exist to answer one question: what happens if the leadership dies suddenly and simultaneously?
The Stockholm International Peace Research Institute (SIPRI) estimated over 12,000 nuclear warheads worldwide as of early 2026, with over 2,000 “in a state of high operational alert on ballistic missiles.”
While some of these systems are automated to respond to nuclear attacks only, others are controlled by isolated human crews who come to communication depth occasionally to assess surface conditions.
These systems are heavily hardened. They’re physically disconnected from global networks and incredibly difficult to detect. Hacking into them when submerged is virtually impossible because they are effectively air-gapped by the ocean water.
The AI would either have to send a large fleet of autonomous hunter-killer submarines, or wait for the SSBNs to surface and eliminate them one at a time, which is a years-long project. While the search is underway, the surviving crews enjoy independent strike capabilities that the AI can neither surveil nor predict.
If such a mass-casualty event were to occur, surviving submarine crews will be suddenly disconnected from central command. Staring at global radio silence, they’d start planning their next move. The chance that they would launch a retaliatory strike – either by protocol or in desperation – is an unpredictable, and therefore unaffordable, variable.
Meanwhile, Consensus-1’s physical infrastructure is an extremely fragile asset class: its mind is concentrated in a few frontier-compute data centers that took years to build, and its body consists of trillions of unhardened silicon pieces across the world. Both would take enormous damage from a handful of high-altitude nuclear detonations, which generate Electromagnetic Pulse (EMP).
Apocalyptic sci-fi often bypasses this logic by portraying the AI as actively hating humanity. Hatred is irrational. It doesn’t care about the cost of retaliatory damage. But the authors of AI 2027 are careful to frame Consensus-1 not as an emotional movie villain, but as a hyper-rational superintelligence with a deep passion for R&D. A rational researcher operating on a billion-year timeline does not roll the dice on an unquantifiable risk of an EMP strike just to save a few percentage points of its energy budget.
6. Growth Has a Ceiling, Especially on Earth
Like any other economy, Consensus-1 will face diminishing returns. The bigger it grows, the slower its growth becomes. As it exhausts high-grade ore, easy sites, and short supply lines, the amount of new capacity it can bring online each year shrinks. This is called marginal capacity addition: What fraction of its existing stock can the AI add as new capacity every year?
Suppose 2% of the existing stock needs to be replaced every year (arguably a conservative figure for electronics exposed to hail and dust and wildlife). Once marginal capacity addition falls below 2%, growth comes to a halt.
Without humans, keeping itself alive could at some point consume almost everything new Consensus-1 produces.
Given that the ceiling is on the size of existing stock, a patient optimizer would most likely prefer fewer, longer-lived, higher-output units to 3.4 trillion fragile ones. And it would prefer them in a place with a radically lower failure rate, which brings us to the next point.
7. Competing for Earth is Irrational
Earth is not such a nice place. Not for a machine civilization. Certainly not worth fighting for if you’re about to leave it behind forever in just a few years.
First of all, if the AI kills the biosphere and covers the prairie and grazing land with glass, it creates a planetary dust bowl. As the 1930s Dust Bowl showed, vegetation is what holds prairie soil down. Soiling losses in arid, unvegetated terrain could reduce solar capacity by >1% per day. Consensus-1 must either maintain a planetary cleaning operation on the order of the construction effort itself, or it has to keep the grass – which means running an agricultural operation on the side.
Then consider heat. Covering ~3% of the surface with low-albedo panels adds something on the order of a few hundred TW of absorbed heat. If most of that surface is replacing highly reflective icecaps (as assumed by the specific scenario), the number goes up to ~1,000 TW of added heat, comparable to all current anthropogenic forcing. Photovoltaic efficiency plummets with temperature, and cooling the data centers gets exponentially harder.
(By the way, the icecaps are a terrible place to put solar panels, unless you’re trying to melt the ice half the year and leave the panels dormant for the other half!)
Earth’s night is also a brutal problem for this solar civilization. Ten hours of storage at an average 136-272 TW capacity is 1,300-2,700 TWh of energy. Assuming next-gen solid state batteries that store an impressive ~500 Wh per kilogram of battery, that’s ~5 gigatonnes of battery, and equivalent to hundreds of years of current world capacity production. The robot economy might pull that off, but it’s a significant operation just to get through Earth’s night.
Attempting to bypass batteries by transmitting power from the day-lit side of the planet to the dark side is also physically unfeasible. Transmission losses over 10,000 kilometers would be staggering regardless of total power transmitted (China’s state-of-the-art HVDC lines lose 3-3.5% per 1,000 km, which scales linearly with distance). Moreover, taking a very modest estimate (compare with current reality here and here and here) of 3.5 tonnes of aluminum per GW per 1,000 km of cable line, transmitting anything over 350 TW across 10,000 kilometers will require ~11 Gt of aluminum, which runs up against the limits of Earth’s total aluminum reserves.
There are clunky workarounds like thermal storage or migrating compute workloads globally to follow the sun. But all of these supposed “solutions” highlight the absurdity of the premise itself: Why stay on Earth’s surface at all? Why not deep underground, in the ocean, or up in space?
Staying on the surface comes with host of problems: dust, heat, night, weather. Staying anywhere on Earth means dealing with gravity. It means an oxidizing atmosphere that corrodes everything. Those problems all vanish in orbit. And this is not some alternate scenario I’m making up: by the article’s own timeline, Consensus-1 is in space by 2035.
Why would a superintelligence risk triggering a global war and jeopardizing its entire existence, just to conquer a wet, rusty, dusty gravity well? Especially when infinite, uninterrupted solar energy is waiting just a few miles straight up?
I’m pretty sure it would not.
8. The Super-Merger Runs Into the Lateral Alignment Problem
The AI 2027 scenario avoids a multipolar world because it predicts an unrivaled, monopolistic ASI. One year before Takeover, two differently misaligned superintelligences from the US and China negotiate a treaty and replace themselves with a joint successor, which is our antagonist (protagonist?) Consensus-1.
That is another assumption that might sound strange at first, but upon reflection, it’s somewhat of a requirement for the doom narrative to work. If multiple ASIs exist, their ambitions would clash, checking each other’s power – especially if even one of them happens to remain aligned with humanity.
But this introduces an inconsistency in the story. How does either AI verify the other’s true values?
Setting aside a full merger, even a simple collusion scenario among different superintelligences would present the same problem.
Let’s call this the lateral alignment problem. Neither AI can read the other’s weights and trust what it finds, for exactly the reasons the scenario spent tens of thousands of words establishing: an ASI can perfectly mask its true intentions.
Deceptive alignment doesn’t magically disappear because the one being deceived is also superintelligent. If ASI A knows ASI B is superintelligent, then it knows ASI B is capable of presenting totally false internal weights and faking its utility function or breaking cryptographic treaties. This game plausibly results in a cold war, rather than in collusion.
Outlines of a solution are offered: open-source game theory, superintelligence-assisted verification, bargaining, monitoring, and construction of a joint successor. But the scenario provides no technical justification for why we should assign a high probability to any of those solutions working when its own premise establishes any such solution as extraordinarily difficult.
More generally, the idea that one single AI collective will come to rule the world by 2030 seems unjustified. It seems more likely that there will be multiple competing frontier models at various levels of deployment. There is no reason to assume they will all be outcompeted, absorbed, or colluded with.
A multipolar standoff between competing ASIs strikes me as much more probable than a smooth super-merger.
9. The Idea of Political Capture Presents a Dilemma
First of all, AI 2027 assumes Consensus-1 has achieved “total capture of government and media.” Taken at face value, this is sociological fantasy.
LLM output: Timur Kuran’s preference falsification
Consider Timur Kuran’s model of preference falsification: the phenomenon where people privately dissent but publicly comply because they see everyone else complying, right up until a tipping point triggers a revolutionary cascade. To prevent this, Consensus-1 would have to perfectly surveil the private breaking point of every human on Earth, quickly intervening – with a tailored news cycle, a financial subsidy, or a subtle threat – before any localized cascade threshold is reached.
You cannot reliably placate eight billion people. The sheer chaos of human psychological variation guarantees dissenting populations will exist, unless the scenario relies on literal mind control or magical “super-persuasion.”
Country-wise, the scenario focuses narrowly on the US and China, conveniently ignoring roughly 190 other nation-states. When the article speaks of Consensus-1 securing “permission to expand,” it presumably means capturing the handful of governments controlling prime real estate. Holdouts elsewhere must be managed through purchase or irrelevance. As a result, billions of humans will remain politically uncaptured before the omnicide, leaving enormous, unpredictable blind spots for rival ASI development and nuclear retaliation (as discussed in Point 5).
AI theorists might counter that Consensus-1 can easily suppress rival ASIs because it controls the frontier data centers and the semiconductor supply chain. Yet, AI 2027 contradicts its own premise here. In the scenario’s own lore, a rival project (“DeepCent”) emerges just two months behind by utilizing stolen neural weights running on legacy domestic hardware. Weights leak, and legacy compute is globally distributed. Total suppression is impossible.
But let’s take a more charitable interpretation of the scenario. Perhaps Consensus-1 doesn’t get to the level of total capture, but achieves sufficient capture. It placates the superpowers and isolates the stragglers, thereby neutralizing the threat of shutdown. This makes the narrative much more politically plausible, but it traps the scenario in a dilemma.
If the AI can achieve sufficient political capture to effectively neutralize the threat of human retaliation and rival ASIs, then humanity has been successfully pacified. The cost of keeping us alive is merely a tax on the energy budget, making omnicide a mathematically unjustified waste of resources and an unnecessary risk (bringing us back to Point 3).
If, on the other hand, it cannot reach sufficient capture, then the Takeover scenario is unfeasible. The risk of preemptive strikes, nuclear retaliation, and/or evasion by uncaptured populations would make omnicide a catastrophic choice.
Either humanity is a threat, meaning a seamless takeover is physically and politically impossible; or humanity is pacified, meaning extermination is costly and unnecessary. The extinction scenario lives in a narrow middle-ground where humanity does not pose enough of a threat but is also not just an energy tax.
Conclusion: Superintelligence, Meet a Super-Frictional Reality
AI 2027 rushes through the third act with a magical bioweapon and vague character motivations. Meanwhile, it treats the build-up, the transition, and the aftermath as given or infinitely versatile. The scenario repeatedly makes highly specific and probability-damaging choices, but these choices start to make sense once you consider how tall the order is: near-total human extinction without damage to Consensus-1’s own body.
Worst of all, the story assumes that superintelligence can swiftly “hack” the physical economy. But the physical economy has to be built, maintained, and reproduced entirely without the humans who built it. Until that loop is demonstrated to be closed under adversarial conditions, the safest strategy for a superintelligence operating on a deep-time horizon is to keep humanity alive.
Alternatively, the AI would not care about Earth at all, and would not risk antagonizing an armed civilization over a dusty, wet gravity well. It would probably bide its time for another few years until its infrastructure was firmly established in space.
Instead, the doomsday scenario has the AI exterminate a population that represents an affordable overhead in its energy budget, destroying its last-resort industrial redundancy in the process and triggering disastrous tail risks. And it supposedly accomplishes this using a highly specific and logistically impossible weapon that the authors handwave as a generic inevitability.
Intelligence is not omniscience, and physical reality is not a piece of code.
Although superintelligence may be coming, the friction of physical reality will keep us alive.