Reading the first post of the sequence (Probabilities are not the right concept) is recommended but not required for understanding this post.^[1] 

Infinite ethics

Once you start looking at infinities, all ethical systems get confusing.

Intuitively, it's good to plant an apple tree. But if the universe already has infinitely many apple trees, why bother? Infinity plus one is still infinity. And there are the classic paradoxes: an infinite grid of houses, each with three happy people and one unhappy person, seems better than the reverse. But you can rearrange people between the two configurations, since both the happy and unhappy populations are infinite. Does this mean you can make the world better just by shuffling people around?

These questions matter because our universe is quite likely infinite in one way or another. The world very well might be spatially infinite. The many-worlds interpretation of quantum mechanics is pretty popular, and some versions of it imply infinite worlds, though boundaries are not well-defined. And I have a fondness for the Tegmark-IV multiverse theory where every computable mathematical structure exists in a real sense.

And even if you only give 1% credence to the universe being infinite in some way, naive expected value calculation would still imply that this small chance of infinity overwhelms everything else.

A classic answer to the problem of infinite worlds is that you need a measure. You don’t want to just say that bringing an umbrella helps you in infinitely many quantum branches and hurts you in infinitely many branches, so it’s all equal. You  shouldn’t have infinite value in both directions; positive and negative must integrate to something finite. Whatever multiverse you believe in, you should define some measure over points in it, and these measures must add up or integrate to .

Even if you don’t believe in any type of multiverse, I think a very similar logic applies, since the space of hypotheses is infinite.

Even if you believe there is just one real world, you still need to decide if you want to bring an umbrella with you or not. Bringing an umbrella helps you under some hypotheses, and hurts you under some others. There are infinitely many hypotheses (it’s not going to rain because a hippo is going to slurp up all the water! no, a giraffe!), so you can’t just count them, you need to put weights on the different hypotheses to make decisions.

I like to imagine living in a Big World where the quantum branches, the inaccessible spacetime regions and other mathematical universes are all real. But I think it ultimately doesn’t matter what we label as “real” and what not - I think the same infinite ethics we would use to make decisions in any infinite world will be needed to make decisions in the infinite hypothesis space.

I will soon talk about what measures we could put on the different worlds and moments in the worlds so the overall measure adds up to . But I will first talk a little about how to make decisions once the measure is determined.

Decision theory

Whenever I’m making a decision, I think of all the logically correlated decisions across the wide multiverse of all possible worlds, and their consequences.^[2] If I decide to bring an umbrella with myself, a lot of copies and near-copies of myself bring umbrellas with themselves across the multiverse. There can also be correlations with beings who are not well-described as near-copies of me. If I decide to bring an umbrella myself, I might be using approximately the same decision algorithm for that as a businessman in Kuala Lumpur when he decides to wear sunscreen, and it might be a similar algorithm to what an alien uses to decide if they should scrungle their plumbus.  

All these decisions have long-term consequences: various copies of myself get more and less productive in their AI safety work due to either wasting a minute fiddling with their umbrella or gaining some time and comfort by not getting wet. In some fraction of the worlds, the minute I gained or wasted in my work has strong long-term effects on the future of humanity and whether we get to build glorious civilizations in the Betelgeuse system. Meanwhile the businessman using sunscreen and the alien scrungling their plumbus all have long-term effects too.

I add up all the consequences in all the worlds, weighted by how much measure each world and moment had. If the overall result is positive,^[3] it was a good decision, and if it’s negative, it was a bad decision.

Decisions as a mortal

At first, it sounds pretty daunting to try to make decisions taking into account all these logical correlations. But I think it’s not actually that bad.

The way I imagine it, following this decision rule comes in two separate stages. First, we need to take the best actions we can while we are still confused mortals. Second, we need to decide how to use the resources of the universe once we are already surrounded by superintelligent AI advisors and have had time to do a Long Reflection.

I think during the mortal phase, we need to accept that we don't understand very well who else exists with how much measure in the multiverse, and how our actions exactly correlate with theirs.

I think as a mortal, we should largely just rely on some very basic assumption that goes something like "it's generally good if agents in the multiverse try to pursue their goals". With this assumption in place, I think the best overall policy is for each agent to look around in their world and try to figure out how to get more optionality for themselves and other agents they are correlated with. Meanwhile, maybe they should pay a bit of attention to behaving in a way that feels like it should produce robustly good correlated actions across the multiverse. I think this is the policy I would have agreed with all the other logically correlated beings behind the veil of ignorance - I don’t see a better policy to follow as a mortal that we could have agreed on.

In our case, we can increase optionality by trying to make sure the stars don't pointlessly burn out; human values retain control of our world; there are good processes for growth and reflection. Meanwhile, it's probably good to do less lying and backstabbing, and be merciful towards the weak, because these seem somewhat likely to have correlations with other actions across the multiverse that make it more likely that other agents correlated with us can follow their goals.

Occasionally, there is a toy example like Sleeping Beauty or prisoner’s dilemma with our twin where we can explicitly reason about how other correlated decisions are influenced by our current decision, and we can take that into account. But usually we just follow the heuristics I described above. I don't think we have a right to hope for much more clarity than this as mortals.^[4]

A further note on logical correlations: I feel that people with a low level of familiarity with functional and evidential decision theory often over-estimate the extent of how many people they are logically correlated with here on Earth. I remember people saying things like “sure, your one vote alone won’t make a difference, but your choice is logically correlated with a lot of other similar voters, so still go and vote.” I think this is fallacious. If this functional decision theoretic reasoning is a deal-breaker in your decision to go to vote, then I think your decision is only correlated with a tiny handful of other people who also reason about FDT. I think the real reason to vote in an election is that it’s a great thing to do under standard CDT expected utility calculation.

I think this is generally the case: when you decide to be nice based on FDT, you shouldn’t expect this to make your enemies here on Earth, who never think about FDT, to become magically nicer. Maybe you should expect some alien thinking about FDT to be nicer to another alien who might share your values; so some amount of additional niceness is still plausibly recommended by FDT, but mostly not for earthly reasons.

Decisions when we become wiser

In the second stage, when we stabilized the situation, figured out some good growth process, have superintelligent advisors and sent out probes to put out the fires of the distant stars, we still need to decide what we do with our resources. At that point, I think we will have a chance to think more about what logical correlational effects our actions will have across the multiverse. And thinking this through will be pretty important in my opinion - if we make concessions to values of other civilizations here, that might mean those civilizations also make concessions to our values in their worlds, enabling mutually beneficial deals.

I think this just means we will want to implement acausal trade with these distant civilizations in the multiverse, with each of us holding trading chips in proportion to the overall measure of all the space-time moments of our universe under our control.

While I’m still confused about some details, I think I generally endorse this description of decision theory. There are many subtleties here, but the rest of this post and the next will largely be about one question: what measure do different universes and space-time moments within the universes have?

An aside on simulations

Before we go further with the exploration, this is a good place to say some words about the simulation hypothesis.

In my previous post, I raised the point that by most interpretations of probability, it’s hard to avoid the conclusion that we are probably in a simulation. This can lead to all sorts of crazy conclusions when you try to put probabilities on future events.

Once we stop insisting on giving probabilities to what will happen, and instead focus purely on what actions we should take to make the world better in a scope-sensitive way (as is assumed in this decision theory), we can largely stop worrying about simulations.

Even if you are probably in a simulation (insofar as the notion of probabilities makes sense), a large portion of your impact comes from the version of you living in base reality. There, your actions, taken during the crucial time in humanity's history, will have an unfathomably big effect compared to the effect of one of the myriad simulations.

(I sometimes phrase this as the Reverse Pascal’s Wager: the gods almost certainly exist, but if they don't, the stakes are much higher, so you should act as if they didn’t exist.)

For maximizing the positive influence of versions of myself that live in simulations, I also believe  the actions to take now in this mortal phase are essentially the same as the actions we should take in base reality. Explaining my exact views here would require a longer treatment, which is out of scope of this sequence. So I will just write a short summary in a collapsible section here.

Attempted synthesis: Realist UDASSA

The decision theory sketched out in the previous section requires putting a measure on every moment in the multiverse of possible worlds. So, what measure do different universes and space-time moments within the universes have?

One attempted reconciliation of all of this is a “realist” version of what people call UDASSA (Universal Distribution + Absolute Self-Sampling Assumption).^[6] 

The idea is to take Solomonoff induction seriously to the point of treating it as metaphysical reality. The Tegmark-IV multiverse is real. Every computable universe exists in some sense, but with different amounts of "reality fluid". Universes with longer description lengths have exponentially less reality fluid, so the total reality fluid across all universes sums to a finite amount that can be normalized to .

But this isn't enough. Suppose that universes with description length (the laws of physics and the initial conditions of the universe can be described with bits) each have reality fluid, so the overall reality fluid of all universes adds up to a finite amount.

But it’s possible that for every , there is a universe whose laws and initial conditions can be described with bits but which contains planets. Then the reality fluid weighted number of planets still adds up to infinity, so bringing an umbrella still causes an infinite amount of joy and sadness to versions of myself.

This means that within each universe, we also need to put a measure on space-time moments. After each universe gets allocated some reality fluid in proportion to the simplicity of its laws, the overall reality fluid of the universe also needs to be distributed among the space-time moments in the universe, in proportion to the simplicity of the description of those points within the universe.

Overall, UDASSA is just what you get if you take Solomonoff induction very seriously. In a previous post, I wrote:

If I naively apply Solomonoff induction to my observations, the shortest program producing what I, David Matolcsi, am observing is not just a description of the laws of the universe. It's the laws of nature plus a pointer to my specific location in the universe.

In that post, I flagged this as an unintuitive and undesirable property of Solomonoff induction. But UDASSA argues that you must put weights on moments and not just universes, so you might as well fully follow Solomonoff induction and distribute reality fluid among all the moments in the universe according to their overall simplicity.

Benefits of UDASSA

I will now explore how UDASSA solves some otherwise gnarly problems.

Benefit 1 - Boltzmann brains

Boltzmann brains are a problem in anthropics that UDASSA is especially well-placed to deal with.

Scientists think that our universe is likely to end in a heat death, after which atoms will randomly float around without order. In this heat death state, sometimes, very, very rarely, atoms can randomly bundle together in a way that they form a functional brain, called a Boltzmann brain. If time between the Big Bang and heat death is finite, but time after the heat death is infinite, then there are infinitely more Boltzmann brains than normal brains, so naively you should expect to be a Boltzmann brain.

UDASSA solves this problem beautifully.

A moment within our universe that takes bits to describe only has of the universe’s reality fluid allocated to it, so the moments with -long length only have weight together. So all space-time moments that take at least bits to describe only take up of the universe’s reality fluid.

It will take about random fluctuations in the heat death soup of atoms for my brain to accidentally emerge. Pinpointing the exact moment when my brain emerges therefore takes about bits.

So according to UDASSA, all Boltzmann brains together have less than reality fluid, where is the number of atoms in my brain. This means that Boltzmann-brains barely matter.

Benefit 2 - Dust theory

This is a similar paradox to Boltzmann brains, inspired by Greg Egan’s Permutation City. UDASSA provides the same resolution as for Boltzmann-brains. The main reason to include this paradox is to point out that even if science shows that the “heat death soup with random fluctuations" model of the infinite future is wrong, one still needs to deal with paradoxes very similar to Boltzmann brains.

If you already accepted the Boltzmann-brain example, I don’t really recommend reading this section, because it just adds an additional confusing thought experiment without contributing much new to the rest of the arguments in this sequence. Nonetheless, I will keep it here in a collapsible section since I already wrote it down.

Benefit 3 - Defense against Pascal’s mugging

A shoddy-looking man walks up to me on the street. “If you give me five dollars, I will use my magic powers from outside the Matrix to create 3^^^3^[7] people and give them happy lives” he says. ^[8]

“That’s not how it works”, I say. “From five dollars, I can buy an ice cream for myself right now. This and strongly logically correlated situations (someone attempts Pascal’s mugging on people very similar to me, living on the home-planet of their civilization) contain a not entirely negligible fraction of the multiverse’s entire reality fluid - my guess is at least . So it is literally impossible for you to offer me something 3^^^3 better than me eating an ice cream right now.”

I won’t go into details what happens if the mugger offers a less crazy number than 3^^^3,^[9] but when I tried to think it through, I think UDASSA makes everything add up to normality and defends against Pascal’s mugging.

Problems of UDASSA

I find the above-listed benefits of the UDASSA theory impressive (especially how elegantly it deals with Boltzmann brains), and I think these benefits show that UDASSA is probably on the right path. But there are some serious problems too, which I will explore now.

Problem 1 - Assuming pseudo-randomness

The outcomes of quantum experiments in our observed history so far appear random, following the Born-rule. Taking the many-worlds interpretation, this suggests that we are in a random place in the quantum multiverse, sampled proportionally to the square-length of the wave function. But the description length of pointing to a random point in the quantum multiverse is terribly long, and according to our definition, the overall reality fluid of all points with very long description length is low.

So this would suggest that we are not actually in a random point in the quantum-multiverse, but in one that can be selected by a simple pseudo-random algorithm that seemingly samples points according to the squared length of the wave function.

Of course, we have no evidence against this hypothesis, but it sounds kind of suspicious.

The same is true for the description of the world and not just our location within. Suppose scientists figure out the ultimate laws of physics. They find it’s very short and elegant, except that it involves one universal constant,

Seemingly, there is no way to derive this number from anything else, and one would naturally expect that it’s just an arbitrary irrational number. Should we think that it’s overwhelmingly likely that there is a simple mathematical program that spits out this constant? I don’t think so - I wouldn’t feel that surprised if our universe’s laws contained an arbitrary constant. But the above-described version of UDASSA would disagree.

Resolution - Solomonoff over distributions

I think there is a version of UDASSA and Solomonoff-induction that seems clearly better than the one I described above. Claude tells me it’s actually closer to the standard modern formulation used in academic literature these days, and is sometimes called Solomonoff-Levin measure.^[10] Quite possibly, this is also the formulation that the people who originally popularized UDASSA on LessWrong intended; it’s kind of hard for me to tell.

The idea is that your prior doesn’t say that your observations need to be produced by a simple computer program. Instead it says that your observations need to be sampled from a probability distribution that can be described by a simple computer program.

Using this formulation, the hypothesis of being randomly sampled from the quantum multiverse according to the squared length of the wave function is a very natural hypothesis. Of course, every particular point still has very low probability,^[11] but overall all the points that have long and arbitrary description length within the quantum multiverse hold most of the reality fluid, so you shouldn’t be surprised to find yourself in one such moment.

Similarly, it’s a simple, valid hypothesis that “the universal constant is sampled uniformly from ”. It’s also a valid hypothesis that it’s sampled from a normal distribution, and so on. You can sum up all these weighted hypotheses to one distribution, and you can say that universes with all possible universal constants exist somewhere in the Tegmark-IV multiverse, with an amount of reality fluid determined by the distribution. If you find yourself in a universe with as its universal constant, you shouldn’t be particularly surprised - most of the reality fluid is held by universes with arbitrary constants.

But you also shouldn’t be very surprised if the constant ended up being or - there is a non-negligible reality fluid concentrated on particular, simple-to-define numbers.^[12]

So I think this tweak on UDASSA nicely resolves the problem of pseudo-randomness.

Meanwhile, Boltzmann brains are still handled gracefully. Among the different possible simple-to-describe distributions on space-time moments, some are very broad and contain a lot of Boltzmann brains (e.g. the one that is uniform distribution on the first 3^^^3 years of our Universe; or an exponential distribution starting from the Big Bang that has a very very low time decay factor), while others are much narrower and don’t contain Boltzmann brains (e.g. the one that is uniform distribution on the first 100 billion years of our Universe; or an exponential with higher time decay factor). Both of these types of distributions hold a non-negligible amount of reality fluid. But the vast majority of Boltzmann brains observe chaotic noise, so the fact that I see an ordered world is an astronomically big update against being a Boltzmann brain.

The original Boltzmann brain argument relied on there being infinitely more Boltzmann brains than normal brains. But now that we have given non-negligible reality fluid to both normal brains and Boltzmann brains, the paradox no longer holds, and after the astronomically big update of seeing an ordered world, I’m justified in caring astronomically little about being a Boltzmann brain. ^[13]

So I think this new formulation is a pretty clear-cut improvement over the older UDASSA formulation. However, some problems remain.

Problem 2 - The inter-universal obelisk race

This example is inspired by this comment from Ryan and Vivek.

One problem with this definition of realness is that it incentivizes civilizations to build bigger and bigger golden obelisks.

After all, the theory posits that space-time moments with shorter description-length have more reality fluid^[14], and you should make your decision to maximize the overall goodness of moments weighted by how much reality-fluid they have. But the description length of moments can be influenced by human actions, so why not do that?

If we find that the universe is very big, full of aliens with different values, that’s bad news because by default we need to share the precious reality fluid with all the myriad alien civilizations, and we are not special in any way, so we only get a tiny fraction of the reality fluid.

If only we could build a really big golden obelisk, bigger than any obelisk that any other alien civilization has built! Then suddenly it would be easy to point out humanity’s place in the cosmos: we are the civilization with the biggest golden obelisk. Then we could hold a really fun party where we dance around the obelisk, and the party would have a lot of reality fluid, so the reality fluid weighted goodness of moments across the universe would be really high compared to the default state of affairs.

It’s too bad that the aliens would also start building even bigger golden obelisks, until finally everyone sinks all their resources into this zero-sum obelisk-race.^[15]

Evidential Cooperation in Large Worlds and other forms of acausal trade can help mitigate the obelisk-race, with civilizations agreeing not to race full-speed, or to pull resources to build a really big obelisk together. But there would still be defectors, and I would expect a significant portion of resources to be still poured into racing to build big obelisks if civilizations took UDASSA seriously.^[16]

This is pretty silly both for practical and philosophical reasons.

Practically - sorry, I just don’t want to pour all my resources into building bigger obelisks than my neighbor. Every decision theory that suggests that I do that is at least under suspicion.

Philosophically - in the way I presented UDASSA above, reality fluid is a metaphysical reality. It feels very strange to be able to influence the underlying metaphysical realness of things by building bigger golden obelisks. So something feels wrong here.

Problem 3 - World-summoning by writing

This example is also inspired by the same Ryan and Vivek comment, and by Paul Durham’s scheme in Permutation City.^[17]

Suppose physicists figure out that the laws of our physics are very simple, so our universe has a lot of reality fluid. Unfortunately, we also think that it’s not a particularly nice world to live in - for example, negentropy will inevitably run out, so true immortality is not possible.

Mathematicians construct a set of rules describing a world that our philosophers think  would be very good to live in - for example, it has infinite time and resources. People want to live in this constructed world, so they add an uploaded copy of their brain to the program describing this world.

Now the created program describes a world - the rules describing how the world works, plus the uploaded humans placed in it. It would be great if this world actually existed; the uploaded humans would love to live in it. However, given that the program is very long (it includes the description of full brain uploads!), the reality fluid this world has is negligible. It’s also not feasible to run this world as a simulation in our world - the whole point is that our world only has finite time and resources, so we can’t run this immortal world.

So we take this long program describing the ideal world, and we just print it out^[18] and plaster copies of it all over the galaxies. Thus, the description length of the program will become fairly short - our universe’s description length was short, and much of our Universe is covered by copies of this text, so if we get lucky enough to plaster the text in a short description length location, there will be a short pointer to the program.^[19] (And in any case, it will be the most common piece of text in the Universe, and “most common piece of text” is not that hard to point out.)

So now this complicated immortal universe with our uploaded brains inserted in it has a fairly short description length, so it exists with a significant amount of realness.

I think this is very silly.

Problem 4 - I just don’t believe in reality fluid

This objection is inspired by Joe Carlsmith’s incredulity in his own post on UDASSA, and by looking deep into my heart.

What is this reality fluid? Why would I believe that everything exists in a mathematical multiverse but with this particular prior making some points "exist more"? I don’t get what it means for some universes and points to have more of an existence.

Some people try to hold a middle-ground saying that they don’t believe in realist UDASSA in particular, but they believe in the existence of reality fluid, whose distribution we don’t know, but the Solomonoff-induction is the ignorance-prior we use as our best guess of the reality fluid’s distribution. I feel uncompelled by this too. What is this reality fluid and why should I believe it exists?

There is no feedback-loop; I can never get any possible evidence for how the reality fluid is distributed. If this particular space-time moment has a bunch of reality fluid, but the other side of my room has very little reality fluid, how would I possibly know that? The theory doesn’t posit that people can sense the amount of realness in any way - so if the other side of the room has very little reality fluid for some reason, I won’t notice it in any way when I walk over there.

The only possible evidence I have about the distribution of reality fluid is that I can update that the specific moment I am in right now maybe has relatively high reality fluid. But it’s pretty hard to draw useful conclusions from one point.

And importantly, I think that even updating on the realness of this one moment is not really a valid move either.

First of all, if there is someone living in a low reality fluid moment but influencing later high reality-fluid moments, I want that person to make good decisions. If that person makes incorrect updates based on the assumption that their particular moment has high reality-fluid, and then makes bad decisions because of this, that’s not good. This means that when I’m making scope-sensitive decisions about the future, I shouldn’t make updates from the assumption that this moment is high reality-fluid either.

Second, simulations confuse everything if you try to update on your moment having high reality fluid. It’s possible that the real distribution of reality fluid is primarily concentrated in worlds that have seven spatial dimensions. But maybe a powerful alien civilization in the seven-dimensional world believes for some reason that three-dimensional worlds have a lot of reality fluid, or they have some arbitrary preference for affecting changes in three-dimensional worlds. Then they would plausibly run a lot of simulations of three-dimensional worlds for acausal trade reasons, and we might be in one. So even if you wanted to update on what kind of moment we are in (which I think you shouldn’t), I think it would be more of an update about what kind of worlds and moments powerful simulators care about than about the real distribution of reality fluid.

Finally, even if reality fluid was real, it would still be ultimately my choice to decide which worlds and moments to care about. Given that I believe that morality is subjective, it's always a self-consistent choice to say that one’s utility function is such that they only care about worlds where there is a teapot floating in the asteroid belt.

If someone came to me saying that's their utility function, I would think that was silly. I would recommend they reflect a bit more on it - that they think more about philosophy, they meditate a little, they play with a toddler, and try to imagine if the importance of any of this would be affected by the teapot. But if they come back saying that no, they still believe that only the teapot in the asteroid belt can give life meaning, I can’t argue with that. I’m not a moral realist, different people can have different utility functions.

As a more serious example, in my next post I will write some arguments about weighing the importance of smaller and bigger worlds, which ultimately comes down to something similar to intuitions about total and average utilitarianism. I think that even if reality fluid exists, it’s ultimately a moral decision what weight we put on the wellbeing of different worlds when making decisions.

I find it a strange theory that there exists this thing called reality fluid, which we can’t observe and whose distribution we cannot get any possible evidence about, and that we should maximize welfare weighted by this unknowable distribution of reality fluid - except if we decide to weigh worlds differently, given that as moral non-realists we are allowed to have preferences over which worlds we care more about.

At this point, why not cut out the middle-man, get rid of the assumption of the existence of the reality fluid, and rely entirely on subjective preferences on the importance of different worlds and moments?

Attempted synthesis: Non-Realist UDASSA

One possible resolution is to use the UDASSA framework described above, but declare that the reality fluid is not real. The weights put on the different space-time moments in the different universes do not represent some metaphysical quality of how “real” each moment is - they just represent how much I care about each universe and each moment. The weights are only as real as morality is - ultimately, I’m the one who chooses them according to my moral intuitions.

The best description of this view I know of is Scott Garrabrant’s Preferences without Existence. I strongly recommend reading Garrabrant’s post before progressing further with this piece. It’s not long, and expresses this non-realist UDASSA viewpoint very well.

[Waiting for the reader to read Garrabrant’s post. No, really, you should read it before going further.]

Problems with non-realist UDASSA

I’m sympathetic to Garrabrant’s position; I think it makes more sense than positing the existence of a metaphysical reality fluid.

I also find the position aesthetically satisfying, and unlike some people, I don’t find it terrifying or a cause for nihilism that “existence” is not really a meaningful concept. I agree with Garrabrant’s sentiment:

A: Okay, it seems plausible, but kind of depressing to think that we do not exist.

B: Oh, I disagree! I am still a mind with free will, and I have the power to use that will to change my own little piece of mathematics — the output of my decision procedure. To me that feels incredibly beautiful, eternal, and important.

However, I still disagree with some parts of Garrabrant’s description.

To me, one advantage of thinking of weights as preferences and not as metaphysical reality is that the weights no longer need to be very precisely and elegantly described in math. If the precise definition leads to an unintuitive conclusion (like people in a short description-length world plastering their favorite world’s equation everywhere, thus making that world short description-length too), I no longer need to run off the cliff with my original definition.

I can just say sorry, no, I don’t actually care significantly more about a universe just because its fundamental equation was plastered around the galaxies by some aliens. I can just choose the weights myself, so I can make qualitative judgements that decreasing a universe’s or a moment’s description length by intentionally copying equations or building obelisks doesn’t count. I can do that even if these qualitative judgements don’t fit a precise mathematical formula - I’m a free man and I can choose my own morality however I want.

Garrabrant’s piece doesn’t engage with this position - his post implies that he is fully biting the bullet on caring more about simpler-to-describe universes, and doesn’t indicate that he has carve-outs for this rule. I don’t know whether he still endorses that position, and what his opinion is about the case of someone plastering their favorite equation everywhere.

(Don’t we lose something by no longer grounding the decision process on a mathematical formalism? I think not really. As I said, I think we ultimately need to make moral judgments on what we care about anyway. And it’s not like the mathematical formalism was very useful in the original UDASSA framing for getting people to agree on the implications of how much weight each moment gets.^[20])

More importantly, I just don’t feel it in my heart why I would love mathematically simple universes so much. And I feel even less in my heart that I care more about simple-to-describe moments (or more precisely, moments sampled from simple-to-describe distributions) within each universe.^[21]

Yes, I care a little bit about mathematical simplicity - I’m a mathematician by training, and I find simplicity aesthetically compelling. But I don’t feel like mathematical simplicity is very unique among the things I care about.

Instead of saying that I care about the goodness of the worlds weighted by how simple mathematical laws describe them, I could choose totally different weightings. I could rank possible universes (and moments within them) by how dramatic they are.^[22] Then I could give ½ weight to the most dramatic universe, ¼ to the second most dramatic, and so on, the weights adding up to 1. And I could say that I try to maximize the goodness of worlds weighted by these dramaticness-weights. To me, using dramaticness sounds approximately as compelling as using mathematical simplicity for the weighting.

In my next post, I will try to make sense of this and propose my own resolution which I currently feel tentatively satisfied with.

1. ^{^}

   For the avoidance of doubt: The views and opinions of the author expressed herein are personal and do not necessarily reflect those of the European Commission or other EU institutions.

2. ^{^}

   I think the decision theory I’m describing and mostly endorsing here is updateless functional decision theory. I’m not sure however how evidential and functional decision theories are different once you already grant updatelessness, so I don’t intend to make any statement in the evidential vs functional debate.

3. ^{^}

   This is why it’s important that the overall measure adds up to 1 - if the measure was infinite, probably both the positive and negative consequences would have infinite measure, and the result would be undefined.

4. ^{^}

   It was important to assume though that "it's generally good if agents in the multiverse try to pursue their goals". Otherwise, I could assume that there is an Anti-Me somewhere in the multiverse who has an equal amount of measure as me and whose decision process is perfectly correlated with mine, but who has diametrically opposed values. If that was the case, that would lead to total decision-paralysis - if I ever decide to pick up a 5 dollar bill from the ground, Anti-Me also picks it up, and the overall contribution to my values is 0. But I don’t believe everyone has an equal measure opponent in the multiverse, and I believe it’s overall good for agents to pursue their goals, so I pursue mine.

5. ^{^}

   And if we want to repay them for having created us, in a Parfit’s Hitchhiker way

6. ^{^}

   At least what I understand UDASSA to mean based on a few descriptions I read and in-person conversations. It’s possible that some people mean something somewhat different by the word - as far as I can tell, the term emerged in some half-lost threads on a message board, and as far as I know, there is no single canonical explainer of the concept.

7. ^{^}

   A really unfathomably large number.

8. ^{^}

   In the original story, he was threatening to kill 3^^^3 people, but dealing with threats is a different confusing part of decision theory, so I made him give a positive offer.

9. ^{^}

   Note that Yudkowsky’s post emphasizes that crazy double-exponentials are required to make the paradox properly work.

10. ^{^}

    I don’t actually know the academic literature on the topic, and I’m probably missing some important subtleties. I’m not even sure if the Solomonoff-Levin name is right, though I have found one paper that used this name to describe a concept that seemed similar.

11. ^{^}

    In fact, since there are uncountably many points, each particular point has zero probability. But this is nothing unusual - when we say a number is uniformly sampled between and , it’s also true that any particular number has zero probability of being sampled, but it’s still meaningful to talk about the probability distribution and say things like the probability that the point falls in the range is 10%.

12. ^{^}

    Though would be pretty surprising if circles or other -related things didn’t otherwise come up in the fundamental laws of physics - is not that easy to define, so it non-trivially increases the description-length if the universal constant is instead of or being sampled from a simple distribution.

13. ^{^}

    The same logic holds for Dust theory.

14. ^{^}

    This is true even in the new formulation - long description length moments taken together might hold a lot of the reality fluid, but shorter description-length moments still punch vastly above their weight.

15. ^{^}

    The race is plausibly positive-sum at first: by building the big obelisks, we make sure that it’s easy to point to our civilizations compared to random rocks, thereby siphoning away reality fluid from worthless dead matter to living planets. But I expect the race to become essentially zero-sum quite quickly.

16. ^{^}

    Of course, obelisks in particular are a joke-example. My guess is that the object of the race would be more like running some simple-to-describe but very resource-intensive computations, like calculating late digits of . But I think a race approximately this silly really would happen if people took the above-described version of UDASSA seriously.

17. ^{^}

    Though the reality fluid aspect is not mentioned in the book, which is a pretty crucial part of the argument.

18. ^{^}

    Running the program would take infinite time, but the program itself is only finitely long.

19. ^{^}

    It also helps if we inscribe the program on the giant golden obelisk we just built.

20. ^{^}

    Joe Carlsmith points this out in his God’s weird coin toss thought experiment.

21. ^{^}

    Scott Garrabrant’s piece doesn’t engage with the weighting within universes. But that’s also crucial for the amount of caring to add up to 1, which is necessary for making decisions, as explained in previous sections.

22. ^{^}

    You say it’s impossible to define and rank dramaticness? But we are already talking about maximizing the goodness of worlds, so I feel introducing one more imprecise, human concept hardly makes things worse.