In the second paper, you mention radical negative utilitarians as a force that could be motivated to kill everyone, but similar considerations seem to apply to utilitarianism in general. Hedonistic utilitarians would want to convert the world into orgasmium (killing everyone in the process), varieties of preference utilitarianism might want to rewire everyone's brains so that those brains experience maximum preference satisfaction (thus effectively killing everyone), etc.

You could argue that mere destruction would be easier than converting everything to orgasmium, but both seem hard enough to basically require a superintelligence. And if you can set the goals of a superintelligence, it's not clear that one of the goals would be much harder than the other.

You could argue that mere destruction would be easier than converting everything to orgasmium, but both seem hard enough to basically require a superintelligence.

We can kill everyone today or in the near future by diverting a large asteroid to crash into Earth, or by engineering a super-plague. Doing either would take significant resources but isn't anywhere near requiring a superintelligence. In comparison, converting everything to orgasmium seems much harder and is far beyond our current technological capabilities.

Could the Maxipok rule have catastrophic consequences? (I argue yes.)

by philosophytorres 1 min read25th Aug 201732 comments

6


Here I argue that following the Maxipok rule could have truly catastrophic consequences.

Here I provide a comprehensive list of actual humans who expressed, often with great intensity, omnicidal urges. I also discuss the worrisome phenomenon of "latent agential risks."

And finally, here I argue that a superintelligence singleton constitutes the only mechanism that could neutralize the "threat of universal unilateralism" and the consequent breakdown of the social contract, resulting in a Hobbesian state of constant war among Earthians.

I would genuinely welcome feedback on any of these papers! The first one seems especially relevant to the good denizens of this website. :-)