I’m going to need an entire separate book, which is mostly just John Ray quotes.

The real superiority of chopsticks becomes clear for foods like Cheetos, or better yet, Flamin’ Hot Cheetos.

Now, sadly, we probably should never eat Cheetos. But that’s just how the cookie crumbles (onto the keyboard).

Eliezer Y, along with I’m guessing a lot of people in the rationalist community, seems to be essentially a kind of Humean about morality (more specifically, a Humean consequentialist). Now, Humean views of morality are essentially extremely compatible with a very broad statement of the Orthogonality Thesis, applied to all rational entities.

Humean views about morality, though, are somewhat controversial. Plenty of people think we can rationally derive moral laws (Kantians); plenty of people think there are certain objective ends to human life (virtue ethicists).

My question here isn’t about the truth of these moral theories per se, but rather: does accepting one of these alternative meta-ethical theories cast any doubt on the Orthogonality Thesis as applied to AGI?

This is basically just a more explicitly AGI-related version of the Fermi Paradox but:

1.If AGI is created, it is obviously very unlikely that we are the first in the universe to create it, and it is likely that it was already created a long time ago.

2.If AGI is created, aligned or unaligned, there seems to be consensus that some kind of ongoing, widespread galactic conquest/control would end up constituting an instrumental goal of the AGI.

3. If AGI is created, there seem to be consensus that its capabilities would be so great as to enable widespread galactic conquest/control.

4. Therefore, we should expect that there is already some kind of widespread galactic control spreading outward caused by AGI.

[4] does not seem to be discussed widely, from what I've seen in this sphere (though I don't really belong to the rationality/LW world in any meaningful way). Why not? Shouldn't people that accept all of the above wonder about A. why we don't see any evidence of [4] and/or B. the implications of encountering [4] at some point?