This is a special post for quick takes by Cleo Nardo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
It’s kinda embarrassing that, if we’re asked how ASI would kill literally everyone, we talk about this how this is an unfair question or smth smth vingean uncertainty smth smth Magnus Carlson. I also don’t think “engineered pandemics” will stand up to scrutiny.
Here’s my list, ranked from most to least likely:
Robots that humans gave them
Robots they built themselves
Nanotech
Mirror life
Super persuasion
Hacking into nukes/critical infrastructure
Weird physics / magic
Blackmailing/hiring humans to kill other humans
Note that the actual route might involve a mix of these, e.g. blackmail someone into giving you compute, hack into a robot factory, capture enough robots to build mirror life, kill enough humans to disempower governments, build robots to build robots, industrial explosion, more foom, nanotech. So this list is more like “what’s the primary vector for how things initially went unrecoverable”.
I also don’t find credible “AIs unintentionally kill humans because they make the world uninhabitable as a biproduct of industrial explosion”. If AI kills all the humans it will be because they didn’t want to risk us turning them off or building a rival RSI.