Hi folks. As some of you know, I've been trying to write an article laying out the simplest case for AI catastrophe. I believe existing pieces are worse than they could be for fixable reasons. So I tried to write my own piece that's better. In the end, it ended up being longer and more detailed than perhaps the "simplest case" ought to be. I might rewrite it again in the future, pending feedback.

Anyway, below is the piece in its entirety:

___

The world’s largest tech companies are building intelligences that will become better than humans at almost all economically and militarily relevant tasks.
Many of these intelligences will be goal-seeking minds acting in the real world, rather than just impressive pattern-matchers.
Unlike traditional software, we cannot specify what these minds will want or verify what they’ll do. We can only grow and shape them, and hope the shaping holds.
This can all end very badly.

The world’s largest tech companies are building intelligences that will become better than humans at almost all economically and militarily relevant tasks

The CEOs of OpenAI, Google DeepMind, Anthropic, and Meta AI have all explicitly stated that building human-level or superhuman AI is their goal, have spent billions of dollars doing so, and plan to spend hundreds of billions to trillions more in the near-future. By superhuman, they mean something like “better than the best humans at almost all relevant tasks,” rather than just being narrowly better than the average human at one thing.

brown wooden hallway with gray metal doors

Photo by İsmail Enes Ayhan on Unsplash

Will they succeed? Without anybody to stop them, probably.

As of February 2026, AIs are currently better than the best humans at a narrow range of tasks (Chess, Go, Starcraft, weather forecasting). They are on par or almost on par with skilled professionals at many others (coding, answering PhD-level general knowledge questions, competition-level math, urban driving, some commercial art, writing¹), and slightly worse than people at most tasks².

But the AIs will only get better with time, and they are on track to do so quickly. Rapid progress has already happened in just the last 10 years. Seven years ago (before GPT2), language models can barely string together coherent sentences, today Large Language Models (LLMs) can do college-level writing assignments with ease, and X AI’s Grok can sing elaborate paeans about how it’d sodomize leftists, in g...

Graygood

Graygood

Graygood

Graygood