This post is somewhat niche, and I will sometimes not give context or link relevant background.
There’s a big debate that has played out in slow motion on LessWrong over the past two decades, between two broad ways of putting a measure over all possible realities (Tegmark IV):
Some “objective” prior (a “reality fluid”), usually a simplicity prior[1]: This is the position taken by Max Tegmark, Jürgen Schmidhuber and UDASSA.
A “caring measure”, where we say that our preferences determine our probabilities and maybe even what counts as “existing”. For example, Wei Dai here, Paul Christiano here and Scott Garrabrant.
These both have significant drawbacks:
A simplicity prior seems to imply some very counterintuitive things, like caring about people more the easier we can find them in the universe (and even weirder things, see David Matolcsi here and Joe Carlsmith here), and is partially dependent on an arbitrary choice of implementation (e.g. which Universal Turing Machine to use in UDASSA).
A caring measure just seems a bit unmotivated - intuitively, our probabilities (or existence itself) shouldn’t entirely depend on our preferences.[2]Ideally, we’d like something better.
Unfortunately, there are infinite possible worlds and every event happens infinitely many times - so we do need some kind of measure to calculate probabilities and the effects of our actions.
Or do we?
Recently, I came across Toby Ord’s “Evaluating the infinite” paper from last year. I wrote about my reaction to it here, and here’s Ord’s Twitter summary - the gist of it is that using hyperreal numbers (where infinity + 1 does not equal infinity) to evaluate infinities in various fields is actually more promising and coherent than people previously thought.
I think this paper might have gone under the radar a bit. I couldn’t find any discussion of it on LessWrong, for example.
More than the specific hyperreal formalism, my main takeaway was more philosophical - a sort of “scales falling from my eyes” / “paradigm shift” realization that the ontology of “infinity + 1 = infinity” never really made sense in the first place. I can’t even remember why I believed it for so long, like it was just an unexamined assumption that immediately collapsed when I thought about it for a moment.
Here’s a quote from an email exchange with Ord that I found useful (reproducing it with permission):
Part of the problem is that maths education has now instilled in many of us the counterintuitive principles of Hilbert’s hotel and the cardinal numbers, and that while these are a very useful concept of infinity, they are not the right one for this job and so our current mathematical intuitions are leading us more astray than if we were mathematically naive.
It does feel to me now like the default way of thinking should be that infinity + 1 > infinity. Like… obviously if you add 1 to something it becomes bigger?
Let’s take this back to the measure problem. If we take this semi-philosophical stance seriously, why do we even need a measure?
Could we just sum up all the infinite possible occurrences of our possible next inputs, normalize, and get probabilities about what our next input will be that way? Sum up everything that we care about across the infinite possible realities, and get estimates of the effects of our actions that way?
That sounds kind of insane. But the more I think about it, the more it feels like the only principled approach. It’s weirdly very grounded. Like, literally just add up everything? Details TBD?
That’s really all I wanted to get across in this post - that this approach seems to be very neglected among thinkers in this area. It very much doesn’t obviously fail, and nobody seems to have seriously thought about it or tried to work out its implications.[3]
It’s far more complicated and ambitious than the simple settings where Ord rigorously showed hyperreals work, and there’s other possible number systems where infinity + 1 > infinity (like surreal numbers), so I want to distinguish this from his specific hyperreal formalism. Let’s call it the measureless approach, for lack of a better term.
Speculatively, we might even recover some version of a simplicity prior from it, since simple worlds might reoccur more frequently across all possible computations, i.e. influence the infinite sum more.
That said, it also has some serious issues (although for me personally, not enough to outweigh its appeal). For example:
It doesn’t solve the puzzle of why we aren’t Boltzmann brains like UDASSA famously does.[4]
It seems like any formalism where infinity + 1 > infinity will give different results for an infinite sum depending on the order of the summands.[5] So this immediately leads to extreme ambiguity about which order to sum all possible realities in. (Although this might be just another way we recover some version of a simplicity prior, since plausibly the only principled order is one given by some UTM like in UDASSA - which would be a nice convergence).
But if some version of a simplicity prior is recovered, this might also recreate some of its issues.
So it’s definitely plausible that it will turn out to not make sense. But the existing approaches don’t seem clearly better!
So the measureless approach seems underrated to me.
This post is somewhat niche, and I will sometimes not give context or link relevant background.
There’s a big debate that has played out in slow motion on LessWrong over the past two decades, between two broad ways of putting a measure over all possible realities (Tegmark IV):
These both have significant drawbacks:
Unfortunately, there are infinite possible worlds and every event happens infinitely many times - so we do need some kind of measure to calculate probabilities and the effects of our actions.
Or do we?
Recently, I came across Toby Ord’s “Evaluating the infinite” paper from last year. I wrote about my reaction to it here, and here’s Ord’s Twitter summary - the gist of it is that using hyperreal numbers (where infinity + 1 does not equal infinity) to evaluate infinities in various fields is actually more promising and coherent than people previously thought.
I think this paper might have gone under the radar a bit. I couldn’t find any discussion of it on LessWrong, for example.
More than the specific hyperreal formalism, my main takeaway was more philosophical - a sort of “scales falling from my eyes” / “paradigm shift” realization that the ontology of “infinity + 1 = infinity” never really made sense in the first place. I can’t even remember why I believed it for so long, like it was just an unexamined assumption that immediately collapsed when I thought about it for a moment.
Here’s a quote from an email exchange with Ord that I found useful (reproducing it with permission):
It does feel to me now like the default way of thinking should be that infinity + 1 > infinity. Like… obviously if you add 1 to something it becomes bigger?
Let’s take this back to the measure problem. If we take this semi-philosophical stance seriously, why do we even need a measure?
Could we just sum up all the infinite possible occurrences of our possible next inputs, normalize, and get probabilities about what our next input will be that way? Sum up everything that we care about across the infinite possible realities, and get estimates of the effects of our actions that way?
That sounds kind of insane. But the more I think about it, the more it feels like the only principled approach. It’s weirdly very grounded. Like, literally just add up everything? Details TBD?
That’s really all I wanted to get across in this post - that this approach seems to be very neglected among thinkers in this area. It very much doesn’t obviously fail, and nobody seems to have seriously thought about it or tried to work out its implications.[3]
It’s far more complicated and ambitious than the simple settings where Ord rigorously showed hyperreals work, and there’s other possible number systems where infinity + 1 > infinity (like surreal numbers), so I want to distinguish this from his specific hyperreal formalism. Let’s call it the measureless approach, for lack of a better term.
Speculatively, we might even recover some version of a simplicity prior from it, since simple worlds might reoccur more frequently across all possible computations, i.e. influence the infinite sum more.
That said, it also has some serious issues (although for me personally, not enough to outweigh its appeal). For example:
So it’s definitely plausible that it will turn out to not make sense. But the existing approaches don’t seem clearly better!
So the measureless approach seems underrated to me.
The other major alternative is Schmidhuber’s speed prior.
They will to some extent, of course - see the ADT paper and Wei Dai here, or David Matolcsi’s Probabilities are not the right concept for the most exhaustive treatment that I know of.
Perhaps because of the quasi-ideology of cardinals that Ord talks about.
I think this is not as bad as it first seems - I will try to write my next post on this.
I think this is more natural than it first seems - see the 2nd edit in my post on Ord’s paper.