Cross-posted from the EA forum (link).

I often want to talk about this idea and I don’t have a good phrase for it. “BC” is an attempt to invoke Cunningham’s Law with a potentially poor term.

Definition

Binary Consequentialism: an ethical position that there are only two possible outcomes for the future, one of which is morally desirable and one of which is morally undesirable.

Actions are ethical to the extent that they make the morally desirable outcome more likely. That is, they maximize p(good).

A trivial form of binary consequentialism would be “the only thing that matters is whether the Patriots win the Super Bowl next year.” Actions are moral insofar as they make the Patriots more likely to win.^[1]

Plausibility

Most forms of BC would be unappealing. A binary utility function seems almost farcical. Surely we care about more than just one outcome?

There are at least two related lines of reasoning that could lead robust existing moral frameworks to a binary consequentialism in practice.

(1) Infinite outcomes dominate finite outcomes in aggregative consequentialism

Infinitely positive outcomes are theoretically possible. The expected value of any finite outcome is dominated by possible infinite outcomes. Infinite outcomes can be dominated by infinite outcomes of a higher order. All of the value of the future is, in expectation, in the worlds where we obtain the highest-order positive infinity of moral value.^[2]

Someday someone will have to reckon with the vicissitudes of infinite ethics. Perhaps fortunately for us, we have no control over whether the density of positive value that tiles infinite time and/or space is N or 2N or 1e35N, so we are spared the challenge of valuing these possibilities.^[3] Such outcomes will only be the result of a maximally capable aligned optimization process whose specific values and abilities we can only guess at.

From our current location in history, the only relevant question is whether such an optimization process comes to exist. Either it will, or it will not, and everything else is not even a rounding error.

(2) The future will either converge on eutopia or diverge

The most valuable possible futures are far, far more valuable than merely pretty-valuable futures. Eutopia is not easy.^[4] This holds across a variety of consequentialist assumptions.

If we end up in a world that captures eve...

Posts

Wikitag Contributions

Comments