I am asking a single question: Is there (or can we build) a morality that can be derived with logic from first principles that are obvious to everyone and require no Faith?
I think it's one of Yudkowsky's better articles. (On a tangential note, I'm amused to find on re-reading it that I had almost the exact same reaction to The Golden Transcendence, thou... (read more)
It's not a binary distinction. If an identical copy was made of one mind and tortured, while the other instance remained untortured, they would start to differentiate into distinct individuals. As rate of divergence would increase with degree of difference in experience, I imagine torture vs non-torture would spark a fairly rapid divergence.
I haven't had opportunity to commit to reading Bostrom's paper, but in the little I did read Bostrom thought it was "prima facie implausible and farfetched to maintain that the wrongness of torturing somebody wo... (read more)
That doesn't resolve quanticle's objection. Your cutoff still suggests that a reasonably individualistic human is just as valuable as, say, the only intelligent alien being in the universe. Would you agree with that conclusion?
Perhaps you've already encountered this, but your question calls to mind the following piece by Yudkowsky: No Universally Compelling Arguments, which is near the start of his broader metaethics sequence.
I think it's one of Yudkowsky's better articles.
(On a tangential note, I'm amused to find on re-reading it that I had almost the exact same reaction to The Golden Transcendence, thou... (read more)