We triggered some other kind of apocalypse - nuclear war, bioweapons, something like that - and it was enough to roll back progress but not wipe out humanity. With the delay and abrupt shifts, people managed to come up with something better than what we have now. The "AI arms race" requires significant infrastructure to be economically viable, and the classic post-apocalypse scenario doesn’t exactly involve training neural networks on supercomputers.

Maybe people had more time (and 0 regulations) for genetic experiments and eugenics (which are simpler than supercomputers even in a post-apocalyptic world), or they realized the destructiveness of Moloch and learned to coordinate (hahaha), or something else entirely.

(If you want to minus, please, do, but write why, I don't bite. If you're more into stories, here's mine called Places of Loving Grace).

It may sound confusing, because I cannot put a 30 minutes post into a comment, so try to steelman it, but this is how it can look. If you have questions or don't like it, please, comment. We can build Multiversal Artificial Static Place Intelligence. It’s not an agent, it’s the place. It’s basically a direct democratic multiverse. Because any good agentic ASI will be building one for us anyway, so instead of having a shady middleman, we can build one ourselves.

This is how we start: we create a digital copy of Earth and make some wireless cool brain-computer-interface armchairs. Like the one Joe and Chandler from Friends had. You can buy one, put in your living room, jump in, close your eyes and nothing happens. You room and the world is exactly the same, you go drink some coffee, you favorite brand tastes as usual. You go meet some friends and you get too excited by the conversation when you cross the road and a bus hits you (it was an accident, the bus driver was a real human, he chose to forget he was in a simulation and was really distraught).

You open your physical eyes in your room, shrug and go drink some water, because you are thirsty after that coffee. The digital Earth gives us immortality from injuries but everything else is vanilla familiar Earth. Even my mom got interested.

Of course we’ll quickly build a whole multiverse of alternative realities, where you can fly and do magic and stuff, like we have a whole bunch of games already.

So I propose we should build eHeaven 1st, eGod 2nd if he’ll be deemed safe after all the simulations of the futures in some Matreshka Bunker. We should make the superintelligence that is a static place first, where we are the only agents. Else we’ll just make an entity that is changing our world, and be changing it too fast and on too big a scale and it will make mistakes that are too big and on too big in scale, because it will need to simulate all the futures (to build the same democratic multiversal simulation with us as his playthings or else exploit some virtual agents that feel real pain) in order to know how not to make mistakes. We don’t need a middleman, a shady builder. It didn’t end well for Adam and Eve, Noah.

I recently wrote a few posts about it and about aligning agentic AIs (it’s much harder but theoretically possible, I think). Purpose-built tool-AI is probably fine. We also have unaligned models in the wild and ways to make aligned open source models unaligned, we’ll have to basically experiment with them in some Matreshka Bunkers like with viruses/cancerous tissue and create “T-cell” models to counteract them. It would’ve been much smarter to vaccinate our world from agentic AIs, then to try to “treat” the planet that we already infected. Wild world we’re heading towards, because of the greed of some rich powerful men. I propose outlawing and mathematically blocking agentic models in code and hardware of course, before some North Korea has created a botnet that spreads dictatorships or something worse.

Do we really want our world to be a battleground of artificial agentic gods? Where we’ll be too small and too slow to do much, we cannot even deal with tiny static and brainless viruses, they escape our labs and kill millions of us.

We can make the place of all-knowing but we should keep becoming all-powerful ourselves, not delegating it to some alien entity.