There's one thing history seems to have been trying to teach us: that the contents of the future are determined by power, economics, politics, and other conflict-theoritic matters.

Turns out, nope!

Almost all of what the future contains is determined by which of the two following engineering problems is solved first:

How to build a superintelligent AI (if solved first, everyone dies forever)
How to build an aligned superintelligent AI (if solved first, everyone gets utopia)

…and almost all of the reasons that the former is currently a lot more likely are mistake theory reasons.

The people currently taking actions that increase the probability that {the former is solved first} are not evil people trying to kill everyone, they're confused people who think that their actions are actually increasing the probability that {the latter is solved first}.

Now, sure, whether you're going to get a chance to talk with OpenAI/Deepmind/Anthropic's leadership enough to inform them that they're in fact making things worse is a function of economics and politics and the like. But ultimately, for the parts that really matter here, this is a matter of explaining, not of defeating.

And, sure, the implementation details of "utopia" do depend on who launches the aligned superintelligent AI, but I expect you'd be very happy with the utopia entailed by any of the possibilities currently on the table. The immense majority of the utility you're missing out on is from getting no utopia at all and everyone dying forever, rather than getting the wrong utopia implementation details.

The reason that the most likely outcome is that everyone dies forever, is that the people who get to impact which of those outcomes is going to happen are mistaken (and probably not thinking hard enough about the problem to realize that they're mistaken).

They're not evil and getting them to update to the correct logical beliefs is a matter of reason (and, if they're the kind of weak agents that are easily influenced by what others around them think, memetics) rather than a matter of conflict.

They're massively disserving everyone's interests, including their own. And the correct actions for them to take would massively serve their own interests as well as everyone else's. If AI kills everyone they'll die too, and if AI creates utopia they'll get utopia along with everyone else — and those are pretty much the only t...

bender

bender

bender

bender