The introspective assessment is what is most persuasive to me here because
(1) it seems like we need some reason for what makes the marginal unit of value (e.g. happy lives, days of happiness, etc.) provide less and less utility marginal unit of utility less and less valuable, independent of the fact that it lets us out of pascalianism and is logically necessary to have a bound somewhere.
(2) bounded utility functions can also lead to counterintuitive conclusions including violating ex ante pareto (Kosonen, 2022, Ch. 1) and falling prey to an Egyptology objection (Wilkinson, 2020, section 6; the post, "How do bounded utility functions work if you are uncertain how close to the bound your utility is?"). But the Egyptology objection may be less significant in practical cases where we are adding value at the margin and can see that we are getting less and less utility out of it, rather than the bound being something we have to think about in advance because we are considering some large amount of value which may hit the bound in one leap (but maybe this isn't so crazy when thinking about AI). And also I guess money pumping is worse than these other conclusions.
(3) bounded utility functions do not seem necessary to avoid pascalianism nor the most obvious option. Someone could easily have an unbounded utility function with regards to sure bets but reject pascalian bets due to probability discounting (or to prevent exploitation in the literal mugging scenario as you mention). But other people bring this up often as a natural response to pascalianism, so I may be missing a reason why you would not want to e.g. value a sure chance of saving 1 billion lives 1,000 times more than saving 1,000,000 lives for sure, but not value a 0.000001 chance at saving 1 billion lives ~at all.
(4) your reasoning makes sense that things cannot get better and better without bound. For a given individual over finite time, it seems like there will be a point where you are just experiencing pleasure all the time / have all your preferences satisfied / have everything on your objective list checked off, and then if you increase utility via time or population, you run into the thing your prior post was about. But if we endorse hedonic utilitarianism, I wonder if this intuition of mine is just reifying the hedonic treadmill and neglecting ways utility may be unbounded, particularly in the negative direction.
I may have indeed made a mistake to frontload the math and thought experiments and put the introspection at the end, rather than centering the introspection and putting the rest in an appendix.
that's not how utility works, utility is the unit of value, and so it doesn't make sense in my ontology to say that they diminish in value.
I don't think I'm anywhere near negative utilitarian enough to empathize with that last point. As I mention in my previous post, I'm quite positive utilitarian.
I don't really have time to digest 2&3 right now, and I find myself confused without reading up on the things you cite.
This is an argument from absurdity against infinite utility functions, but not quite against unbounded ones.
Can you elaborate on the practical distinction? My impression is that if your utility function is unbounded, then you should always be able to devise paths that lead to infinite utility - even by just infinite amounts of finite utility gains. So I don't know if the difference matters that much.
Infinite utility functions mean that there is a concrete input such that the output is "infinity", such as "you go to heaven in the Wager scenario". Unbounded utility functions do not necessarily output "infinity" for a particular value. or "count the number of paper clips" is unbounded but at no concrete input does it tell you "infinity".
If your utillity is finite, how would you determine its bounds?
If the bound is a practical figure, and not up in the ↑↑↑↑-sphere, that looks rather like scope insensitivity. If it is up there, it is indistinguishable from unbounded in the practical realm.
This is very far up, above my hopes for humanity in the good ASI worlds, but not wildly higher than that, I expect. This is not a practical post afaik, and I said so. It is for filling out our conception of utilitarianism, and adding robustness to edge cases can sometimes help with creating useful new frames. Historically, it is the idea that came to me first and inspired me to write the sublinear utility post.
Most[1] problems with unbounded utility functions go away if you restrict yourself to summable utility functions[2]. Summable utility functions can still be unbounded.
For example, if each planet in the universe gives you 1 utility, and for , then your utility function is unbounded but summable. In such a universe it would be very unlikely for a casino to hand out a large number of planets.
Your proof relies on the assumption
assuming that the casino has unbounded utility to hand out.
and this assumption would be wrong in my example.
This seems like it works but demands a very strange universal prior that penalizes big things and large numbers. I consider the original Pascal's Mugging post to have settled the argument about this type of prior.
McGee's argument is akin to the following piece of mathematics, separate from utility theory. What is ?
Clearly, it is equal to , which equals , which is .
Clearly, it is equal to , which equals , which is .[1]
We do not respond to this paradox by supposing there must be a maximum and a minimum integer. We cannot, because mathematics, even more than physics, is a coherent whole in which we cannot change one thing without changing everything. We must instead accept the fact that not all sequences converge.
It may seem like one can answer McGee's paradox by saying "oh well, I guess my utility function's bounded", but coherence problems will still arise, which I was alluding to in asking how you might find the bounds and what sort of magnitude you imagine for them. What happens in someone's quest for maximising utility when they begin to succeed? To approach the maximum possible utility they could ever have? One would have to find oneself caring less and less about every new thing, until one's future is the torpor of "utility death", jaded beyond all caring. This is indeed a standard trope in fiction, but if death is a solvable problem, I would expect utility death to be so also.
By choosing different sequences, one can produce examples where either or both of the limits of the regrouped sequences is finite. ↩︎
For context on how I discuss utilitarianism in this sequence, read the first post.
There is a mathematical proof that is a compelling case for bounded utility functions, but isn't the whole story.
tl;dr: Vann McGee proves that agents with unbounded utility functions and under reasonable assumptions about their epistemics are consistently vulnerable to Dutch Books which exploit their willingness to seek out high-utility low-probability outcomes in some contexts.
Proof Outline
Consider an agent in a world with infinitely many states, and the agent believes that some infinite (not necessarily strict) subset of those states is possible (although they can have zero probability).
If the agent has an unbounded utility function, then you can subject it to a Dutch Book using an infinite sequence of bets about truth values of the propositions :
Bet 1: You lose one util if is true and gain utils if is false.
Bet 2: You lose 2 utils if is false, and you gain utils if is true and is false; otherwise, the bet is called off.
Bet : You lose utils if are all true and is false. You gain utils if are all true and is false. Otherwise, the bet is called off.
Each of these bets has an expected utility of 1, making it advantageous to take them, assuming that the casino has unbounded utility to hand out. However, only finitely many of the bets will be won with any reality-measure, so this scheme leads to the agent always losing net utility if it chooses to take the whole infinite bundle of bets.
McGee goes into more detail on the mathematical nuances of this. Peter De Blanc investigates a more general and abstract angle on this problem.
The above proof implies the trilemma:
I won't be responding to that post here, and I think we can agree to not do the thing labeled "unreasonable epistemics", since 0 and 1 are not probabilities.
This leaves us with option 2, but does it really make sense to have a bounded utility function? I'm going to try and come at this from several different angles, in the hopes of conveying why bounding utility makes sense as a property of coherent agents.
The proof assumes an infinite sequence of bets, which nobody ever has time to execute, so there's some question as to whether this conclusion holds up in real life. For that, I reference this passage from McGee's paper which gives a reason to care about this argument, albeit one that is more poetic than satisfying:
Even a simply infinite sequence of bets is something in which we mortals never have a chance to participate, so as long as our interest in decision theory is purely practical, we needn’t fret over the example, whether it’s presented to us in the static or dynamic version. There is, however, some theoretical interest in trying to devise a standard of rationality so steadfast that one could safely rely on it even if confronted with an infinite array of choices. We only have occasion to make finitely many choices, but it would be surprising if the finitude of the set of choices we make were a prerequisite for rationality.
I don't have an argument that is properly satisfying, but I do have several different suggestive intuition pumps that constitute much of the reason that I think of myself as having a bounded utility function.
There is a classic situation in decision theory called Pascal's Wager:
Some people claim there is a God who will send you to
heaventhe land of infinite utility, if and only if you Believe in Him and perform some specific rituals. You're a good Bayesian, and you don't assign literally zero probability to this God being real. Infinite utility multiplied by a positive probability is infinity, so you should pick up the infinite expected utility on the ground and join this religion.
This argument is generally considered around LessWrong to be Wrong, and if you haven't seen it before I encourage you to look for flaws yourself.
Done looking for flaws? Okay. Here is mine:
This argument proves far too many things to be jointly the optimal action. For every possible action, there is a logical possibility of a god that rewards that specific thing with infinite utility, and so the expected utility of every action is infinite, even the ones we class as obviously stupid, like donating all of your money to Effective Evil. This is an argument from absurdity against infinite utility functions, but not quite against unbounded ones.
Pascal's Mugging is a variant of this that classically goes as follows:
There is a Guy on the street who asks you for $5, threatening that otherwise He will step outside The Matrix and simulate [1] suffering humans.
This steps around the problem of all the infinities being the same, but in doing so it creates several more minor problems and one major one:
If your utility function is bounded, then you aren't exploitable in this way.
The St. Petersburg paradox proposes a game:
I flip a fair coin until it comes up tails for the first time, and I note down the total number of flips (including the final tails) as . After, I pay you $.
The question is, how much should you pay to enter into this game? Once again, I encourage you to work it out if you haven't seen this before.
We can calculate the expected payout as follows:
Hmmmm.
That's weird.
If we're measuring payouts in money, then this runs into the finite funds of St. Peter's Casino, as well as the fact that many agents have sublinear utility in money. If payouts are given in utility, then this runs into the same infinity problem as Pascal's Wager. Among the solutions for both the money version and the utility version is, of course, bounded utility functions.
Outside the realm of thought experiments with mysterious Russian games and interdimensional wizards, it still makes sense to me to bound my utility function. I won't say this is quite "practical" in the sense that I use it for real decisions in my real life, but it does genuinely provide ontological clarity.
One equivalent rephrasing of "my utility function is bounded" is "my utility function has a maximum and minimum value".
This fact isn't quite explained by my concept Sublinear Utility in Population despite that addressing utility functions asymptoting to some fixed value, as mhampton (correctly) notes in a comment (lightly reformatted and truncated):
This applies only to interpersonal aggregation, and so if you can have unboundedly high utility in one individual, your utility function is not truly bounded, right? I.e., it would get you out of Pascal's muggings of the form, "Pay me five dollars and I will create happy copies of Alice" but not of the form "Pay me five dollars and I will create one copy of Alice and give her utils."
When I try to think about getting utils, this doesn't really seem well defined. When I try to construct an ideal utopia or anti-ideal dystopia in my head, this doesn't intuitively look like the scale of goodness keeps going up or down infinitely, it feels like any changes I make are asymptoting towards an optimum, whether it's in making there be more happy people or making one person really really happy, or any of the other things I value.
I model that, if my utility function were actually unbounded, then the process of finding a utopia would feel like always going "hmmm, I could make this world a substantial amount better by just changing this set of details" and always getting returns in utility of similar sizes, rather than running into diminishing returns.
To overuse an example, if my utility function was exactly the number of paper clips in the universe, then each time I would try to imagine a concrete utopia, I would always be able to add more paper clips to get a better world, without ever asymptoting or plateauing in utility, and without reaching an optimum.
Maybe some people's intuitive values are structured like this, in which case they can work things out between them and the unbounded utility paradoxes I've listed here. I don't have a good sense of how many people have introspective assessments of their values matching mine in this respect, but I'm curious to find out.
very very very large number, see Knuth's up-arrow notation for the definition
Not to say that I'm not interested in Very Big Numbers categorically, I'm just not inclined to care when the number of people is so much larger than the number of atoms in the universe.
"But the casino must have finite funds available" and "But I have sublinear utility in money", respectively.