Cross-posted from the EA forum (link).

I often want to talk about this idea and I don’t have a good phrase for it. “BC” is an attempt to invoke Cunningham’s Law with a potentially poor term.

Definition

Binary Consequentialism: an ethical position that there are only two possible outcomes for the future, one of which is morally desirable and one of which is morally undesirable. 

Actions are ethical to the extent that they make the morally desirable outcome more likely. That is, they maximize p(good).

A trivial form of binary consequentialism would be “the only thing that matters is whether the Patriots win the Super Bowl next year.” Actions are moral insofar as they make the Patriots more likely to win.^[1] 

Plausibility

Most forms of BC would be unappealing. A binary utility function seems almost farcical. Surely we care about more than just one outcome?

There are at least two related lines of reasoning that could lead robust existing moral frameworks to a binary consequentialism in practice.

(1) Infinite outcomes dominate finite outcomes in aggregative consequentialism

Infinitely positive outcomes are theoretically possible. The expected value of any finite outcome is dominated by possible infinite outcomes. Infinite outcomes can be dominated by infinite outcomes of a higher order. All of the value of the future is, in expectation, in the worlds where we obtain the highest-order positive infinity of moral value.^[2]

Someday someone will have to reckon with the vicissitudes of infinite ethics. Perhaps fortunately for us, we have no control over whether the density of positive value that tiles infinite time and/or space is N or 2N or 1e35N, so we are spared the challenge of valuing these possibilities.^[3] Such outcomes will only be the result of a maximally capable aligned optimization process whose specific values and abilities we can only guess at.

From our current location in history, the only relevant question is whether such an optimization process comes to exist. Either it will, or it will not, and everything else is not even a rounding error.

(2) The future will either converge on eutopia or diverge

The most valuable possible futures are far, far more valuable than merely pretty-valuable futures. Eutopia is not easy.^[4] This holds across a variety of consequentialist assumptions.

If we end up in a world that captures even 0.1% of the maximum possible future value, it will be because humanity or another aligned civilization^[5] runs a procedure to determine something like CEV^[6] and then applies itself to the achievement of that goal. Worlds where this does not occur are likely to achieve approximately zero percent of available moral benefit.

In our current moment, we are vanishingly unlikely to affect whether that civilization achieves 0.1%, 4%, or 99% of its potential. That could depend on how we handle the black holes at the end of the universe or something even more colossal and alien. However, we may have the ability to affect whether viatopia is achieved at all.

The future then has two apparently discrete outcomes: non-viatopia futures, where no civilizational process is run to converge on the result of (something like) CEV, and viatopia futures where that process does occur. Viatopias will definitionally converge to maximally-good outcomes over time, so it doesn’t make sense to talk about a “partial viatopia.” 

All that matters to us, then, is to maximize p(viatopia).

-

These are very similar (though not precisely equivalent) lines of argument that do and do not account for infinite outcomes. In my opinion the infinitist version is more accessible. It takes a lot of legwork to establish the fragility of a finite eutopia, but this essay is a valiant attempt.

Commentary

BC doesn't require us to place absolute value on something as narrow as the outcome of a football game. It merely requires that over worlds we can directly influence, given our state of knowledge, our utility function is entirely dependent on something which can definably occur or not occur, even assuming substantial moral uncertainty or strong forms of aggregative consequentialism.

Because we're saying that these other ethics converge to what I'll call a "naturalistic" binary consequentialism, we should expect BC to prescribe the same actions. And indeed it does. Preventing human extinction, aligning AGI, expanding the moral circle of civilization, and preventing negative lock-in all remain important.

The immediate difference is only one of framing. If the result of the EV calculations is that number go up most by maximizing the odds of achieving a viatopia, we can "pull forward" that practical objective in our thinking. 

I think different thoughts when I plan to maximize p(viatopia) or p(CEV) and when I plan to maximize something like utility. Conversations with these different framings also tend to be very different. In my experience the BC framing is more intuitive but YMMV.

-

The provided justifications are shorthand at best and include some contentious claims. There are entire literatures being elided. I hope they nevertheless gesture at the ways in which other forms of consequentialism appear to converge on placing maximal value on a singular outcome. My goal in this post is not to provide a complete justification of BC, but to make it easier to talk about the (surprising, to me) practical simplification in some consequentialist ethics when they are extended to extremes. 

To the extent that that simplification is valid I think it’s worth making explicit, especially since this sort of position appears to be increasingly common. 

My optimistic take is that convergence across ethical philosophies and concrete worldviews on BC might be grounds for practical reconciliation and collaboration. 

1. ^{^}

   A number of people subscribe to this particular philosophy, though it remains a minority view.

2. ^{^}

   We may need to consider an infinitely negative outcome created by some inverted optimizer, requiring we extend to a Ternary Consequentialism where we may maximize p(good) - p(bad) while being indifferent to p(the band of outcomes that are rounded to zero). This assumes symmetry between good and bad and now we’ve opened a whole can of worms beyond the scope of this post.

3. ^{^}

   The literature on infinite ethics is certainly not unanimous on this point. Some people (myself included) end up at this position. For a more thorough treatment of infinite ethics, Joe Carlsmith provides a good overview. 

4. ^{^}

   This post is largely inspired by the Better Futures series. The terms eutopia and viatopia make it much easier to talk about the topic at hand.

5. ^{^}

   Civilization is chosen as an inclusive term. The relevant capacity of such a civilization might consist entirely of a superintelligent AI.

6. ^{^}

   CEV is IMO still the best term available for pointing to the good thing but it doesn’t precisely capture the idea of the hard-to-define “best achievable outcome.” I don’t have a perfect term for “the good outcome in a convergent naturalistic binary consequentialism.” Eutopia may be correct, though it has a more specific technical meaning and has the practical drawback of sounding exactly like “utopia.” (Nevertheless I think the coinage of "viatopia" and "eutopia" is excellent.)