Some people think that they have unbounded utility functions. This isn't necessarily crazy, but it presents serious challenges to conventional decision theory. I think it probably leads to abandoning probability itself as a representation of uncertainty (or at least any hope of basing decision theory on such probabilities). This may seem like a drastic response, but we are talking about some pretty drastic inconsistencies.

This result is closely related to standard impossibility results in infinite ethics. I assume it has appeared in the philosophy literature, but I couldn't find it in the SEP entry on the St. Petersburg paradox so I'm posting it here. (Even if it's well known, I want something simple to link to.)

(ETA: this argument is extremely similar to Beckstead and Thomas' argument against Recklessness in A paradox for tiny probabilities and enormous values. The main difference is that they use transitivity +"recklessness" to get a contradiction whereas I argue directly from "non-timidity." I also end up violating a dominance principle which seems even more surprising to violate, but at this point it's kind of like splitting hairs. I give a slightly stronger set of arguments in Better impossibility results for unbounded utilities.)

Weak version

We'll think of preferences as relations $<$ over probability distributions over some implicit space of outcomes $Ω$ (and we'll identify outcomes with the constant probability distribution). We'll show that there is no relation $<$ which satisfies three properties: Antisymmetry, Unbounded Utilities, and Dominance.

Note that we assume nothing about the existence of an underlying utility function. We don't even assume that the preference relation is complete or transitive.

The properties

Antisymmetry: It's never the case that both $A < B$ and $B < A$ .

Unbounded Utilities: there is an infinite sequence of outcomes $X_{1}, X_{2}, X_{4}, X_{8}, \dots$ each "more than twice as good" as the last.^[1] More formally, there exists an outcome $X_{0}$ such that:

$X_{2^{k}} < \frac{1}{2} X_{2^{k + 1}} + \frac{1}{2} X_{0}$ for every $k$ .
$X_{0} < \frac{1}{2} X_{1} + \frac{1}{2} X_{0}$ ^[2]

That is, $X_{1}$ is not as good as a $\frac{1}{2}$ chance of $X_{2}$ , which is not as good as a $\frac{1}{4}$ chance of $X_{4}$ , which is not as good as a $\frac{1}{8}$ chance of $X_{8}$ ... This is nearly the weakest possible version of unbounded utilities.^[3]

Dominance:...