Cross-posted from the EA forum (link). I often want to talk about this idea and I don’t have a good phrase for it. “BC” is an attempt to invoke Cunningham’s Law with a potentially poor term. Definition Binary Consequentialism: an ethical position that there are only two possible outcomes for...
Cross-posted from the EA forum (link).
I often want to talk about this idea and I don’t have a good phrase for it. “BC” is an attempt to invoke Cunningham’s Law with a potentially poor term.
Binary Consequentialism: an ethical position that there are only two possible outcomes for the future, one of which is morally desirable and one of which is morally undesirable.
Actions are ethical to the extent that they make the morally desirable outcome more likely. That is, they maximize p(good).
A trivial form of binary consequentialism would be “the only thing that matters is whether the Patriots win the Super Bowl next year.” Actions are moral insofar as they make the Patriots... (read 788 more words →)
I used to worry that people taking life extension seriously, particularly around these parts, was a bad thing for AI risk. If the people working to build AI believe that death is very bad, that death is technically solvable, that they and their loved ones will die by default, and that building superintelligence in their lifetime is their best shot at longevity escape velocity, then they have a strong incentive to move as quickly as possible. Getting AI researchers to believe more strongly in any of these four ideas has always seemed like a dubious plan at best.
Recently I've changed my mind somewhat, and I now think that longevity research and life... (read more)
CEV is (imo) a good concept for what we ultimately want and/or what humanity should try to achieve, but I've always found it hard to pithily talk about the intermediate world states to aim for now if we want CEV eventually.
I've heard the practical goal discussed as "a world from which we can be as sure as possible that we will achieve CEV". Doesn't really roll off the tongue. It would be nice to have a cleaner shorthand.
The term "viatopia" seems meant to capture the same idea: https://newsletter.forethought.org/p/viatopia
This also seems like the sort of thing Bostrom might have coined a term for fifteen years ago in some obscure paper.
I'd be interested in... (read more)