Preferences without Existence

[-]cousin_it12y160

I've heard that point of view from several people. It's a natural extension of LW-style beliefs, but I'm not sure I buy it yet. There are several lines of attack, the most obvious one is trying to argue that coinflips still behave as coinflips even when the person betting on them is really stupid and always bets on heads. But we've already explored that line a little bit, so I'm gonna try a different one:

Are you saying that evolution has equipped our minds with a measure of caring about all possible worlds according to simplicity? If yes, can you guess which of our ancestor organisms were already equipped with that measure, and which ones weren't? Monkeys, fishes, bacteria?

(As an alternative, there could be some kind of law of nature saying that all minds must care about possible worlds according to simplicity. But I'm not sure how that could be true, given that you can build a UDT agent with any measure of caring.)

3Scott Garrabrant12y

Evolution has equipped the minds in the worlds where thinking in terms of simplicity works to think according to simplicity. (Ass well as worlds where thinking in terms of simplicity works up to a certain point in time when the rules become complex) In some sense, even bacteria are equipped with that, they work under the assumption that chemistry does not change over time. I do not see the point of your question yet.

4Kutta12y

You conflate two very different things here, as I see. First, there are the preferences for simpler physical laws or simpler mathematical constructions. I don't doubt that they are real amongst humans; after all, there is an evolutionary advantage to using simpler models ceteris paribus since they are easier to memorize and easier to reason about. Such evolved preferences probably contribute to a matemathician's sense of elegance. Second, there are preferences about the concrete evolutionarily relevant environment and the relevant agents in it. Naturally, this includes our fellow humans. Note here that we might also care about animals, uploads, AIs or aliens because of our evolved preferences and intuitions regarding humans. Of course, we don't care about aliens because of a direct evolutionary reason. Rather, we simply execute the adaptations that underlie our intuitions. For instance, we might disprefer animal suffering because it is similar enough to human suffering. This second level has very little to do with the complexity of the underlying physics. Monkeys have no conception of cellular automata; you could run them on cellular automata of vastly differing complexity and they wouldn't care. They care about the kind of simplicity that is relevant to their day-to-day environment. Humans also care about this kind of simplicity; it's just that they can generalize this preference to more abstract domains. (On a somewhat unrelated note, you mentioned bacteria. I think your point is a red herring; you can build agents with an assumption for the underlying physics, but that doesn't mean that the agent itself necessarily has any conception of the underlying physics, or even that the agent is consequentialist in any sense). So, what I'm trying to get at: you might prefer simple physics and you might care about people, but it makes little sense to care less about people because they run on uglier physics. People are not physics; they are really high-level constructs

5Wei Dai12y

Suppose I wanted to be fair to all, i.e., avoid "physics racism" and care about everyone equally, how would I go about that? It seems that I can only care about dynamical processes, since I can't influence static objects, and to a first approximation dynamical processes are equivalent to computations (i.e., ignoring uncomputable things for now). But how do I care about all computations equally, if there's an infinite number of them? The most obvious answer is to use the uniform distribution: take an appropriate universal Turing machine, and divide my "care" in half between programs (input tapes) that start with 0 and those that start with 1, then divide my "care" in half again based on the second bit, and so on. With some further filling in the details (how does one translate this idea into a formal utility function?), it seems plausible it could "add up to normality" (i.e., be roughly equivalent to the continuous version of Solomonoff Induction).

0kokotajlod12y

It sounds like this solution is (a) a version of Solomonoff Induction, and (b) similarly suffering from the arbitrary language problem--depending on which language you use to code up the programs. Right?

0Kutta12y

To clarify my point, I meant that Solomonoff induction can justify caring less about some agents (and I'm largely aware of the scheme you described), but simultaneously rejecting Solomonoff and caring less about agents running on more complex physics is not justified.

6Wei Dai12y

I think I understood your point, but maybe didn't make my own clear. What I'm saying is that to recover "normality" you don't have to care about some agents less, but can instead care about everyone equally, and just consider that there are more copies of some than others. I.e., in the continuous version of Solomonoff Induction, programs are infinite binary strings, and you could say there are more copies of simple/lawful universes because a bigger fraction of all possible infinite binary strings compute them. And this may be more palatable for some than saying that some universes have more magical reality fluid than others or that we should care about some agents more than others.

3Scott Garrabrant12y

I agree with this, but I am not sure if you are trying to make this argument within my hypothesis that existence is meaningless. I use the same justification within my system, but I would not use phrases like "there are more copies," because there is no such measure besides the one I that I assign.

5Wei Dai12y

Yeah, I think what I said isn't strictly within your system. In your system, where does "the measure that I assign" come from? I mean, if I was already a UDT agent, I would already have such a measure, but I'm not already a UDT agent so I'd have to come up with a measure if I want to become a UDT agent (assuming that's the right thing to do). But what do I based it on, and why? BTW, have you read my post http://lesswrong.com/lw/1iy/what_are_probabilities_anyway/ where option 4 is similar to your system? I wasn't sure option 4 is the right answer back then, and I'm still in the same basic position now.

3Scott Garrabrant12y

Well in mind space, there will be many agents basing their measures on different things. For me, it is based my intuition about "caring about everyone equally," and looking at programs as infinite binary strings as you describe. That does not feel like a satisfactory answer to me, but it seems just as good as any answer I have seen to the question "Where does your utility function come from?" I have read that post, and of course, I agree with your reasons to prefer 4.

1Scott Garrabrant12y

I address this "physics racism" concern here: http://lesswrong.com/lw/jn2/preferences_without_existence/aj4w

0cousin_it12y

It seems to me that bacteria are adapted to their environment, not a mix of all possible environments based on simplicity. You can view evolution as a learning process that absorbs knowledge about the world and updates a "prior" to a "posterior". (Shalizi has a nice post connecting Bayesian updating with replicator dynamics, it's only slightly relevant here, but still very interesting.) Even if the prior was simplicity-based at the start, once evolution has observed the first few bits of a sequence, there's no more reason for it to create a mind that starts from the prior all over again. Using the posterior instead would probably make the mind much more efficient. So if you say your preferences are simplicity-based, I don't understand how you got such preferences.

0Scott Garrabrant12y

Would you still have this complaint if I instead said that I care about the subset of universes which contain me following simplicity based preferences? To me, at least as I see it now, there is no difference between saying that I care about all universes weighted by simplicity and saying that I care about all universes containing me weighted by simplicity, since my actions to not change the universes that do not contain me.

0cousin_it12y

In the post you described two universes that don't contain you, and said you cared about the simpler one more. Or am I missing something?

2Scott Garrabrant12y

I am saying that my decision procedure is independent of those preferences, so there is no evolutionary disadvantage to having them. Does that address your issue?

[-]VAuroch12y100

A: Wait, if simplicity changes preferences, but does not change the level of existence, how do you explain the fact that we appear to be in a world that is simple? Isn’t that a priori extremely unlikely?
B: This is where it gets a little bit fuzzy, but I do not think that question makes sense. Unlikely by what measure? You are presupposing an existence measure on the collection of theoretical worlds just to ask that question.

I saw a good explanation of this point (in the #lesswrong IRC sometime last month), which I'll try to reproduce here.

Imagine that there is a specific flavor of Universal Turing Machine which runs everything in Tegmark IV; all potential-universe objects are encoded as tapes for this UTM to simulate.

Take a simple set of rules. There is only one tape that corresponds to these precise rules. Now add an incredibly-specific edge case which applies in exactly one situation that could possibly arise, but otherwise have the same simple set of rules hold; this adds another tape that will produce precisely the same observations from the inside as the simple rules, except in one case that is grotesquely uncommon and very likely will never occur (because it happens to req... (read more)

2private_messaging12y

Not quite. There's an infinite number of tapes that correspond to these precise rules exactly. Each "program of length L" is also two programs of length L+1 , four programs of length L+2, and so on - the following bits are simply not read. In general, there's a large number of different ways how a program with same behaviour could be implemented, and for programs up to a specific length, the number is larger for programs that describe fundamentally simpler behaviours.

1VAuroch12y

That only works under some methods of specification of the model machine, and I don't consider it to be a reasonable generalization to assume. It's at best contested, and at worse semantic distinction-without-a-difference.

1private_messaging12y

Should not be a problem for you to name one where it doesn't work, right? Universal TMs all have to permit setting up an interpreter for running the code for another UTM, you really have a lot less lee-way for "methods of specification" than you think. If you are hung up on there being an actual difference in the output, rather than hand-waving about the special circumstances or such, you ought to provide at least a sketch of a set up (so that people can understand clearly how you get more of the simpler programs). For example, for programs that do halt, instead of halting you can make them copy extra bits from the input tape to the output tape - that would be really easy to set up. And the programs in general can be compactly made to halt after something like BusyBeaver(10) steps.

1VAuroch12y

Whether those L+1, L+2, etc. count as different programs from the length-L one is, last I checked, contentious, and because theorists feel strongly about this, various different widely-used formalisms for defining what makes something a UTM disagree about whether unread tape matters. If someone pokes a hole in the argument in the great-grandparent so that the general conclusion works iff the 2 L+1, 4 L+2, etc. count as different universes, then it would become worth addressing. But as long as it works without it, I'm going to stick with the most difficult case.

4private_messaging12y

Again, give an example for the assertions being made. As for your argument, as others have pointed out, you did not prove anything about the extra length for setting up special output in very specific circumstances, or sketched how that could be accomplished. "Sticking with the most difficult case" you got into the region where you are unable to actually produce an argument. It is far less than obviously true (and may well be false) that the programs which are simple programs but with special output in very specific circumstances, are a notable fraction of large programs.

0abramdemski12y

My knowledge of algorithmic information theory marks the approach advocated by private_messaging as "the best way" as established by years of experimenting with ways to specify priors over programs. I admit to little knowledge of the controversy, but agree with private_messaging's insistence for a burden of providing-the-alternative on your part.

1VAuroch12y

What? There is no mention of this anywhere. I have no idea what you're referring to with this phrase. I'm not going to provide an alternative because it doesn't matter. Using the alternate formulation where you count every padded program with the same results as being unique, you get the same results, with the addition of another underlying assumption. By assuming they do not count as the same, I (am attempting to) force potential objections to confront the actual basis for the argument. So go ahead, if you like. Treat those as distinct and work out the consequences. You'll reach precisely the same conclusions.

2kokotajlod12y

I'm not sure about this. This sounds like a version of this, which I agree with. What I'm wary of is this: This sounds like a mathematical claim that isn't yet proven. My intuitions aren't sharp enough to make an educated guess on it either.

1VAuroch12y

It is indeed a version of that. The vanishingly-small edge cases I refer to would be specific descriptions describing something like "At 5:03:05 PM on Tuesday the 7th of March, 3243, if two photons collide at coordinates X,Y,Z, emit three photons each of equal energy to the incoming two in such-and-such an orientation." Something that has an effect at most once during the entirety of the universe, and possibly not at all. This can be specified as an additional behavior in space proportional to the description of the required conditions + space proportional to the description of the required result (or more likely the diff of that result from the ordinary-rules result), which can be very small. And it's obviously true that you can add a huge number of these before any observer could expect to detect enough of them to make a dent in the long-run confidence they had in the correctness of the simple model. You can treat any difference more systematic than this as a different model while still having enough ability to 'make comments' to force the effective Kolmogorov weighting.

4private_messaging12y

Why would that be more common than "create a giant blackhole at same coordinates"? Or an array of black holes, spaced by 10 light years, or the like, you get the idea. You need to establish that little differences would be more common than giant differences I described.

1VAuroch12y

No, I don't, because they don't have to be more common. They just have to be common. I didn't include black holes, etc. in the simple version, because they're not necessary to get the result. You could include them in the category of variations, and the conclusion would get stronger, not weaker. For most observers in the universe, there was always a giant black hole there and that's all there is to it. The set of small variations is a multiplier on the abundance of universes which look like Lawful Universe #N. The larger the set of small variations, the bigger that multiplier gets, for everything.

2private_messaging12y

You've been trying to show that "universes where the correct generalization is the simple rules are more common than universes where the correct generalization is the complex rules", and you've been arguing that this is still true when we are not considering the longer programs that are exactly equivalent to the shorter programs. That, so far, is an entirely baseless assertion. You only described - rather vaguely - some of the more complex programs that look like simpler programs, without demonstrating that those programs are not grossly outnumbered by the more complex programs that look very obviously more complex. Such as, for example, programs encoding an universe with apparent true randomness - done using the extra bits. That being said, our universe does look like it has infinite complexity (due to apparent non-determinism), and as such, infinite complexity of that kind is not improbable. E.g. I can set up a short TM tape prefix that will copy all subsequent bits from the program tape to the output tape. If you pick a very long program at random, it's not very improbable that it begins with this short prefix, and thus corresponds to a random universe with no order to it whatsoever. Vast majority of long programs beginning with this prefix will not correspond to any shorter program, as random data is not compressible on average. Perhaps a point could be made that most of very long programs correspond to universes with simple probabilistic laws.

2VAuroch12y

No, that's not it at all. I have described how, for every complex program that looks complex, you can construct a large number of equally complex programs that look simple, and therefore should expect simple models to be much more common than complex ones.

3private_messaging12y

You'd need to show that for every complex-looking program, you can make >=n simple looking programs, which do not overlap with the other simple looking programs that you're constructing for another complex looking program. (Because it won't do if for every complex looking program, you're constructing the same, say, 100 simple looking programs). I don't even see a vague sketch of an argument for that. edit: Hell you haven't even defined what constitutes a complex looking program. There's a trivial example: all programs beginning with the shortest prefix that copies all subsequent program bits verbatim onto the output tape. These programs are complex looking in the sense that vast majority of them do not have any simpler representation than they are. Those programs are also incredibly numerous. edit2: also the whole argument completely breaks down at infinity. Observe: for every even integer, I can construct 10 odd integers (10n +1, 10n+3, ...) . Does that mean a randomly chosen integer is likely to be even? No.

1VAuroch12y

That is exactly what I've done, and it's sufficient. The whole point is to justify why the Kolmogorov measure for apparent universe probability is justified starting from the assumption that all mathematical-object universes are equally likely. Demonstrating that the number of additional copies that can be made of a simpler universe relative to the more complex one is in direct proportion to the difference in Kolmogorov complexity, which is what I have done, is sufficient.

3private_messaging12y

You know, it'd be a lot more helpful if it was anything remotely close to "done" rather than vaguely handwaved with some sort of fuzzy (mis)understanding of terms being discussed at it's core. What does "difference in Kolmogorov complexity" even mean when your program of length L does not have any equivalents of length <L ? If it has no simpler equivalent, Kolmogorov's complexity is L. Given a program describing some "simple rules" (what ever that means, anyway), one can make a likewise large number of variations where, instead of a single photon being created somewhere obscure or under some hard to reach conditions, photons are created on a randomly spaced regular lattice over some space of conditions, for example, with some specific spacing of the points of that lattice. Which is very noticeable, and does not locally look like any "simple rules" to much anyone. edit: note that most definitions of T.M. do not have pointers, and heads move by 1 step at a time, which actually makes it very nontrivial to do some highly localized, surgical changes to data, especially in the context of some program that's applying same rules everywhere. So it is not obviously the case that a single point change to the world would be less code than something blatantly obvious to the inhabitants.

1abramdemski12y

This doesn't look remotely like a mathematical proof, though.

1VAuroch12y

Who said anything about a mathematical proof? I linked a more formal exposition of the logic in a more abstract model elsewhere in this comment thread; this is an application of that principle.

0kokotajlod12y

private_messaging beat me to the point.

0Scott Garrabrant12y

The amount of space it takes to encode that difference puts a bound on the number of such differences there can be. and they add up to less than, or at least less than a constant times the weight of the original simple model.

1VAuroch12y

For a given description length, yes, but since there is no necessary bound on description length, there is not a necessary limit to the number of possible differences. As your description length devoted to 'comments' increases, you can make the responses and circumstances ever more specified, multiplying the number of worlds which resemble the simpler world, relative to the more complex one.

0Scott Garrabrant12y

Yeah, that comment was when I thought you were talking about a simplicity weighting, not the naive weighting.

0Scott Garrabrant12y

I do not think this works. All the complex worlds which look like this simple world put together (with Kolmogorov weight) should sum up to less than the simple alternative. There are infinitely many ways to perturb a simple set of rules, but they get much more complex very quickly.

1VAuroch12y

The point is that this is a derivation for Kolmogorov weighting. These universes are weighted naively; every universe is weighted the same regardless of complexity. From this naive weighting, after you add the nearly-undetectable variations, the Kolmogorov measure falls out naturally. kokotajlod linked this, which uses the same logic on a more abstract example to demonstrate how the Kolmogorov measure arises. It's significantly better written than my exposition.

0Scott Garrabrant12y

Ok, I was completely misunderstanding you. The claim that we are probably in a complex world makes sense in the naive weighting, but I do not see any motivation to talk about that naive weighting at all, especially since it only seems to make sense in a finite multiverse.

1VAuroch12y

Now I don't understand you. I'll take a stab at guessing your objection and explaining the relevant parts further: This logical derivation works perfectly well in an infinite universe; it works for all finite description lengths, and that continues in the limit, which is the infinite universe. Every "lawful universe" is the center of a cluster in universe-space, and the size of the cluster is proportional to the simplicity (by the Kolmogorov criterion) of the central universe. This is most easily illustrated in the finite case, but works just as well in the infinite one. I do implicitly assume that the naive weighting of 'every mathematical object gets exactly one universe' is inherently correct. I don't have any justification for why this should be true, but until I get some I'm happy to treat it as axiomatic. The conclusion can be summarized as "Assume the naive weighting of universes is true. Then from the inside, the Kolmogorov weighting will appear to be true, and a Bayesian reasoner should treat it as true."

0Scott Garrabrant12y

I think I completely understand your position. Thank you for sharing. I agree with it, modulo your axiom that the naive weighting was correct. All of my objections before were because I did not realize you were using that naive weighting. I do have two problems with the naive weighting: First, it has the problem that description length is a man-made construct, and is dependent on your language. Second, there are predicates we can say about the infinite descriptions which are not measurable, and I am not sure what to do with that.

[-]abramdemski12y80

My personal objection to this seems to be: "But what if some things DO exist?" IE, what if existence actually does turn out to be a thing, in limited supply?

Let me try to unfold this gut reaction into something which makes more sense.

I think what I'm "really" saying is that, for me, this view does not add up to normality. If I were to discover that "existence" is meaningless as a global predicate, and (to the extent that it's meaningful) all possibilities exist equally, I would change the way I care about things.

I think I valu... (read more)

2abramdemski12y

Additionally, it's interesting to note that I treat the idea (that existence has no referent) as a hypothesis, and it's a hypothesis which I believe I can collect evidence for and against: The book "theory of nothing" by Russel Standish contains a derivation of the Schrodinger equation from axioms which are supposed to be purely about an ideal observer and the process of observation. If this argument were convincing, I would take it as strong evidence for the hypothesis: if we can derive the (otherwise strange and surprising) workings of quantum mechanics from the assumption that we're an ideal observer sitting inside of Tegmark IV, that's decent evidence for Tegmark IV. For better or worse, the argument isn't so convincing: my main objection is that it slips complex numbers in as an assumption, rather than deriving them from the assumptions. So, the complex-valued wave function is not explained. There is a derivation of quantum logic from the logic of self-reference, and I consider this to be very weak evidence for the hypothesis. Bruno Marchal does work along these lines, and discusses a world-view with some similarities to what Coscott proposes.

[-]Scott Garrabrant12y60

Thank you all so much for all of your comments.

Three separate comment threads simultaneously lead to the refutation that I seem to be unfairly biased towards agents that happen to be born in simple universes. I think this is a good point, so I am starting a new comment thread to discuss that issue. Here is my counter point.

First, notice that our finite intuitions do not follow over nicely here. In saying that beings in universes with 20 fewer bits are a million times as important, I am not saying that the happiness of this one person is more important tha... (read more)

1kokotajlod12y

I think that Solomonoff Induction already gets us to the conclusion we want, with the problem that it is relative to a language. So one way to put your point would be this: "There is no fact of the matter about which language is 'right,' what really exists is an infinite unordered jumble of universes. In order to think about the jumble, much less describe values over it, we must fix a language with which to describe it. Why not pick a language that favors a certain pleasing kind of simplicity? And hey, if we do this, then thanks to SI it all adds up to normality!" Retreating into the impregnable swamp of infinity may save you here, but it is a dubious move in general. Compare to someone who thinks that they will win the lottery, because they believe in a Big World and thus there are infinitely many copies/futures that will win the lottery. Edit: Thank you, by the way, for this conversation and discussion. I'm very interested in this topic and I like the way you've been thinking about it. I hope we can continue to make progress!

[-]kokotajlod12y60

As I understand it, your attempted solution to the Problem of Induction is this:

(a) Deny that there is a fact of the matter about what our future experiences will be like

(b) Care about things in inverse proportion to the Kolmogorov complexity of the structure in which they are embedded.

This is why it all adds up to normality. Without (a), people could say: Go ahead, care about whatever you want, but under your belief system you ought to expect the world to dissolve into high-complexity chaos immediately! And without (b), people could say: Go ahead, deny th... (read more)

1Scott Garrabrant12y

That sounds correct.

1kokotajlod12y

Okay. I've considered this view for a while now, but I can't bring myself to hold it. I'm mostly OK with step (a), though I still have niggling doubts. Step (b) is the problem. My values/utility function just doesn't work like that. I realize that if I choose my values appropriately, I can make the mathematical multiverse add up to normality. But that's nothing special--give me any wacky set of beliefs about the world, and I can choose values such that it all adds up to normality. I'm having trouble seeing why people in complicated, anti-inductive worlds are less valuable than people in simple, inductive worlds. Maybe they are less beautiful in some abstract aesthetic sense, but they aren't less valuable in the relevant moral sense--if I can help them, I should, and I ought to feel bad if I don't.

0Scott Garrabrant12y

I address this concern here: http://lesswrong.com/lw/jn2/preferences_without_existence/aj4w

0diegocaleiro12y

There should be a an alarm bell when he said "majority of your counterparts" and you accepted that. Regardless, I'm curious because if you truly think all mathematical structures exist shaped and countable in proportion to how many Kolmogorov descriptions account for each, then you should not care about nearly anything and should live in the moment. Since nothing you do will ever change which Kolmogorov descriptions exist or fail to exist., in your deflated conception of existence.

0Scott Garrabrant12y

I was interpreting his rejection to b to be replaced by something else in which "majority of your counterparts" makes sense. I do not agree with this. The counterfactual in which I thought I was agreeing that you should live in the moment was a scenario in which there was some other distribution in which most worlds are complex. (i.e. uniform over all programs expressible)

[-]Squark12y30

Interesting approach. The way I would put it, the number you want to maximize is expectation of U(X) where U is a utility function and X is a random universe taken from a Solomonoff ensemble. The way you put it, the number is the same but you don't interpret the sum over universes as expectation value, you just take it to be part of your utility function.

What I feel is missing in your approach is that U by its nature is arbitrary / complex, whereas the Solomonoff prior is simple and for some reason has to be there. I.e. something would go wrong on the phil... (read more)

0Scott Garrabrant12y

I have exactly the opposite view. I think that the Solomonoff is arbitrary and complex, and therefore I doubt it has any real meaning outside of my arbitrary and complex mind.

[-]moridinamael12y20

So why haven't you tried walking into a casino and caring really hard about winning? I'm not just being a prick here, this is the most concise way I could think to frame my core objection to your thesis.

[-]Squark12y100

This is an incorrect interpretation of Coscott's philosophy. "Caring really hard about winning" = preferring winning to losing. The correct analogy would be "Caring about [whatever] only in case I win". The losing scenarios are not necessarily assigned low utilities: they are assigned similar utilities. This philosophy is not saying: "I will win because I want to win". It is saying: "If I lose, all the stuff I normally care about becomes unimportant, so when I'm optimizing this stuff I might just as well assume I'm going to win". More precisely, it is saying "I will both lose and win but only the winning universe contains stuff that can be optimized".

2Scott Garrabrant12y

I agree with this comment. Thanks.

0Scott Garrabrant12y

That question is equivalent to asking "Why have you not tried just changing your values to be satisfied with whatever the truth is?" I could possibly reprogram myself to have different beliefs, but I do not want to do that. Even if I replace myself with someone else who has different beliefs, I still will not be happy in the future worlds that the present me cares more about. (the simple ones)

0moridinamael12y

I guess I'm a little confused. Would you be less happy because your knowledge that you possess reality manipulation magic would make you disillusioned about the simplicity of your universe? You think you would be less happy with millions of dollars because you knew that you won it through reality manipulation magic, plus verification that you have reality manipulation magic, than you would be persisting in a universe indistinguishable from one which obeys causal laws? Regardless of this misunderstanding, I think you have simply expressed a definition of existence which is meaningless. Or rather, your subjective expectations have nothing at all to do with the quantity that you are describing, but you are conflating them under the same concept-label of "existence." You can't steer your subjective experience into preferred branches of existence merely by wishing it, because you're embedded in a causal universe that just goes on being causal around you without too much regard for your preferences. If you put in a lot of work changing the universe around you to be more suitable to your needs, only then can you steer your subjective experiences toward desirable outcomes. At no point does this look like the future automatically shaping up to be more the way you prefer.

0Scott Garrabrant12y

My point is that I do not change my beliefs to believe I will win the lottery for the same reason that you under your model of the multiverse do not change your preferences to make yourself happy with the world as is. I was trying to communicate that reason, but instead, let me just say that my model does not add any extra issues that you did not already have with the question "Why don't you just try really hard to not care about the things that bother you?" in everyone else's model. I am not understanding the rest of your post either. I am not trying to put anything under the concept-label of "existence." I am trying to say that "existence" is a meaningless concept in the first place. I can't steer my subjective experience, because there is no one future subjective experience. However I can steer the experience of my future self within the simple universes, which is exactly what I want to do.

[-]mwengler12y10

Ever since learning the Anselmian ontological argument for the existence of god, I have tended towards the Kantian idea that existence is not a predicate. A ball that is not red still bounces, a ball that does not exist... doesn't exist. I'll say right up front I am not a philosopher, but rather a physicist and an engineer. I discard ideas that I can't use to build things that work, I decide that they are not worth spending time on. I don't know what I can build using the idea that a ball that does not exist is still a ball. And if I can build somethi... (read more)

[-]torekp12y10

What are the semantics of "exist" on this view? When people say things like "Paris exists but Minas Tirith doesn't" are they saying something meaningful? True? It seems like such statements do convey actual information to someone who knows little about both a French history book and a Tolkein book. Why not just exercise some charitable interpretation upon your fellow language users when it comes to "exist"? We use "existence" concepts to explain our experiences, without any glaring difficulties that I can see (so a charitable interpretation is likely possible). But maybe I've missed some glaring difficulties.

7Scott Garrabrant12y

I think the definition of "exist" in the statement "Paris exists but Minas Tirith doesn't" is "exists within the visible universe." I think that that sentence still has meaning, just as normal. I am rejecting that there is some objective global meaning to existence. "Exists within the visible universe" is a useful concept, and makes sense because it is defined relative to me as an observer.

1torekp12y

Excellent. I favor something like "X will be implied by our best explanation of experience" rather than "X is within the visible universe", but I think broadly speaking, we're in agreement here. But note that this definition of "existence" allows me to dismiss most of Tegmark's Level IV objects as nonexistent. (Your version would allow the dismissal of more.) And of course I'm also free not to share the preference pattern of caring less about events/life-stories in proportion to their complexity.

0Scott Garrabrant12y

I try to justify my preference pattern here http://lesswrong.com/lw/jn2/preferences_without_existence/aj4w

[-]Gunnar_Zarncke12y10

Reminds me of some discussions I had long ago.

The general principle was: Take some human idea or concept and drive it to its logical conclusion. To its extreme. The result: It either became trivial or stopped to make sense.

The reason for this is that sometimes you can't separate out components of a complex system and try to optimize them in isolation without losing important details.

You can't just optimize happyness. It leads to wireheading and that's not what you want.

You can't just optimize religious following. It leads to crussades and witch hunts.

You ... (read more)

4abramdemski12y

These monks still exist. They have been informed.

2Gunnar_Zarncke12y

Sure they know now. But when the monks belief structure formed they didn't know. Now that that is a given they incorporate that in a suitable compartment as the rest is obviously stable.

0abramdemski12y

They (not just the monks-- many of the ordinary practitioners) put filters on water faucets to drink as few bacteria as possible; don't eat root crops because harvesting them disturbs the soil too much; may refuse antibiotics; and in general, do everything they think is within reason to minimize killing. There's probably some compartmentalization, but not a huge amount in that area.

1Gunnar_Zarncke12y

I take this as a piece of factual information. I do not assume that you want to imply some lesson with it.

2abramdemski12y

Yea, I'm just providing additional information because your model of the Jains seemed incomplete. Jainism

0Gunnar_Zarncke12y

Very incomplete. I just relayed the anecdote about the monks to add color. Interesting. Jains texts should be read by vegetarians to put them into perspective I guess.

0Squark12y

Coscott's philosophy is more or less orthogonal to the choice of empathy. E.g. a "Coscottist" might only care about here kin, but she will care more about versions of her kin embedded in simple universes.

0Scott Garrabrant12y

Would you mind elaborating on why this reminded you of those discussions? Is it because in the first half of the dialogue, I am following a chain where I am starting with a belief and slightly tweaking it little by little, and end up with a similar belief in a different circumstance?

0Gunnar_Zarncke12y

No. Because you explored the extremes of empathy toward beings. You don't follow a path going from less to more extreme as I outlined but an extreme in different aspects you explore nonetheless.

3Scott Garrabrant12y

Oh, ok, I get the connection now, thanks. My main point was not that I care about non-existing things in any way that conflicts with cares about existing things. My point was that believing in the concept of existence is not necessary for having preferences. I suppose I would agree with If not for the fact that I don't think existence has any meaning.

[-]Brian_Tomasik12y10

Nice dialogue.

It's true that probability and importance are interchangeable in an expected-utility calculation, but if you weight A twice as much as B because you care twice as much about it, that implies equal probabilities between the two. So if you use a Solomonoff-style prior based on how much you care, that implies a uniform prior on the worlds themselves. Or maybe you're saying expected utility is the sum of caring times value, with no probabilities involved. But in that case your probability is just how much you care.

If we were in a complex world, i... (read more)

3Scott Garrabrant12y

Thanks! I am saying that expected utility is the sum of caring times value, with no probabilities involved. If there are going to be any probabilities involved at all, they will come from logical uncertainty, which is a separate issue. This can be thought of as saying that your probability is just how much you care, which is how I think about it. However, this has some philosophical consequences. It means that probabilities really are completely subjective. It also means that trying to talk about tautologies outside of the context of a person's beliefs is completely a mind projection fallacy. This explains some issues that I have with anthropics, by dismissing them as ill formed questions. This is plausible, but I do not think it is likely to be possible, unless there is a simulation of you in the simple universe, in which case, you should some "probability" to being in that universe. Just choosing actions which correlate with the actions of simpler agents, just by the mechanism of you being similar to that agent. I do not think this will work, because you having the knowledge that your world is the more complex one is enough to make your decision procedures sufficiently different. I might be wrong, but I believe that the Tegmark Level 4 universe does not by his definition come with a prior. When you say this, you are talking about Tegmark 4 with the Solomonoff prior, right?

1Brian_Tomasik12y

I was going to ask how you handle anthropics, but then you answered it. Trippy stuff. If probability is just degree of caring, why would we use Bayes' rule to update? Or are you also proposing not to update? It probably works sometimes for outputs that aren't related to knowledge of how complex your world is. For instance: Consider a simulation of you at the neuronal level making some decision. It produces some output. Then there's another simulation down to the molecular level. It produces some slightly more accurate output. Then another at the quantum level; it's yet again slightly more accurate. If you're the quantum-level one, your output correlates highly with the neuronal-level one, which is much simpler, so you care about it vastly more. Not necessarily Solomonoff, especially since the multiverse of mathematical structures doesn't conform to a Solomonoff distribution. The set of finite or infinite bitstrings is countably infinite, but the set of mathematically possible universes is uncountably infinite, e.g., universes where some parameter is set to each possible real number. I just meant some measure over the universes. If you restrict Tegmark 4 to bitstring universes, then Solomonoff could work.

2Scott Garrabrant12y

I do not update, but in the exact same sense that Updateless Decision Theory does not update. It adds up to the Bayes' rule normality. Again, plausible, but I am really not sure. Either way, I feel like we do not understand nearly enough to have this strategy of sacrificing our own world for other worlds be practical right now. This one of the beauties of my proposal, is that if we do not have to assign probabilities to possible universes, we don't have to limit ourselves to an uncountable infinity. The collection of universes does not even have to be a set!

2Brian_Tomasik12y

Hmm. Seems like your caring-about measure should still sum to 1. If you're just comparing two universes, all you need to know is their relative importance, but if you want to evaluate policies over the whole set of universes, you're going to want a set of weights whose sum is bounded.

4Scott Garrabrant12y

Great point. However, even if the multiverse is infinite, or not even a set, I as a finite mind can only look at finite pieces of it. My caring function looks at a small piece of the multiverse, because it cannot comprehend the whole thing. This is sad. However, it does not feel arbitrary to me. The caring function has limitations of finiteness, or limitations of set theory, but that is MY limit function. There is a big difference to me between me having a limited caring function, and thinking that the universe has built in a limited probability function.

3Brian_Tomasik12y

I see. :) You can do that, and it's psychologically plausible. I'm old-school and still believe there's some fact of the matter about what the multiverse is. Presumably this fact of the matter is representable analytically (though not necessarily by human minds). If we found a better mathematical way to capture this, presumably your limitations would expand and you would then care about more than you do now.

1abramdemski12y

I enjoyed this sentence. Only to a point! Suppose Coscott has a policy for accepting new math which applies a critical eye and only accepts things based on a consistent criteria. That is: if he would accept X upon inspection, then, he would not have accepted not-X via his inspection. Then consider the "completion" of Coscott's views; that is, the (presumably uncomputable) system which is Coscott in the limit of being taught arbitrarily many new math techniques. Now apply Tarski's Undefinability. We can construct a more powerful math which Coscott could never accept. Therefore, if Coscott is a sufficiently careful mathematician, then his math powers seem to have ultimate limits already, rather than mere current limits. On the other hand, if there is no such limit, because Coscott could accept either X or not-X for some X depending on which is presented to him first, then there is hope! A "stronger" teacher could show Coscott the way to the more powerful math. Yet, Coscott is also at risk of being misled. Here's a riddle: according to Coscott, is there any way Coscott could be mislead? Coscott has established that physical existence is meaningless to him; but is there mathematical truth beyond provability? Is there a branch of the multiverse in which the axiom of choice is true, and one where it is false? (It seems clear that there is a subset of the universe where the axiom of choice is true, and one where it is false, but I don't think that's what I mean...)

0Scott Garrabrant12y

What do you mean by "psychologically plausible?"

0Brian_Tomasik12y

I mean it's a plausible way to describe how people actually feel and make decisions when acting.

0wedrifid12y

Your limitation does not inherently prohibit you from having a caring function that looks at the entire multiverse. The constraint is on how complex the pattern of evaluation of the features of the multiverse can be.

0Scott Garrabrant12y

Fair enough. I was aware of that, but did not bother to write it out. Sorry.

[-]Squark12y00

I'm not beginning to think I identified a real flaw in this approach.

The usual formulation of UDT assumes the decision algorithm to be known. In reality, the agent doesn't know its own decision algorithm. This means there is another flavor of uncertainty w.r.t. which the values of different choices have to be averaged. I call this "introspective uncertainty". However, introspective uncertainty is not independent: it is strongly correlated with indexical uncertainty. Since introspective uncertainty can't be absorbed into the utiltity function, indexical uncertainty cannot either.

I have a precise model of this kind of UDT in mind. Planning to write about it soon.

0Scott Garrabrant12y

I guess I should wait for you to write up your UDT model, but I do not see the purpose of introspective uncertainty.

[-]Raiden12y00

Instead of saying that you care about simpler universes more, couldn't a similar preference arise out of pure utilitarianism? Simpler universes would be more important because things that happen within them will be more likely to also happen within more complicated universes that end up creating a similar series of states. For every universe, isn't there an infinite number of more complicated universes that end up with the simpler universe existing within part of it?

0Scott Garrabrant12y

In order to make claims like that, you have to put a measure on your multiverse. I do not like doing that for three reasons: 1) It feels arbitrary. I do not think the essence of reality relies on something chunky like a Turing machine. 2) It limits the multiverse to be some set of worlds that I can put a measure on. The collection of all mathematical structures is not a set, and I think the multiverse should be at least that big. 3) It requires some sort inherent measure that is outside of the any of the individual universes in the multiverse. It is simpler to imagine that there is just every possible universe, with no inherent way to compare them. However, regardless of those very personal beliefs, I think that the argument of simpler universes show up in more other universes does not actually answer any questions. You are trying to explain why you have a measure which makes simpler universes more likely by starting with a collection of universes in which the simpler ones are more likely, and observing that the simple ones are run more. This just walks you in circles.

0Raiden12y

I guess what I'm saying is that since simpler ones are run more, they are more important. That would be true if every simulation was individually important, but I think one thing about this is that the mathematical entity itself is important, regardless of the number of times it's instituted. But it still intuitively feels as though there would be more "weight" to the ones run more often. Things that happen in such universes would have more "influence" over reality as a whole.

0Scott Garrabrant12y

I am saying that in order to make the claim "simple universes are run more," you first need the claim that "most universes are more likely to run simple simulations than complex simulations." In order to make that second claim, you need to start with a measure of what "most universes" means, which you do using simplicity. (Most universes run simple simulations more because running simple simulations is simpler.) I think there is a circular logic there that you cannot get past.

0Raiden12y

Another thought: Wouldn't one of the simplest universes be a universal turing machine that runs through every possible tape? All other universes would be contained within this universes, making them all "simple."

5Scott Garrabrant12y

Simple things can contain more complex things. The reason the more complex thing can be more complex is that it takes extra bits to specify what part of the simple thing to look at.

1Raiden12y

What I mean though, is that the more complicated universes can't be less significant, because they are contained within this simple universe. All universes would have to be at least as morally significant as this universe, would they not?

0Scott Garrabrant12y

If I have have a world containing many people, I can say that the world is more morally significant than any of the individual people.

2Squark12y

I'm not following you here. I think Raiden has a valid point: we should shape the utility function so that Boltzmann brains don't dominate utility computations. The meta-framework for utility you constructed remains perfectly valid, it's just that the "local" utility of each universe has to be constructed with care (which is true about other meta-frameworks as well). E.g. we shouldn't assigned a utility of Graham's number of utilons to a universe just because it contains a Graham's number of Boltzmann brains: it's Pascal mugging. Maybe we should start with a bounded utility function...

1Scott Garrabrant12y

I am not sure if Raiden's intended point is the same as what you are saying here. If it is, then you can just ignore my other comment, it was arguing with a position nobody held. I absolutely agree. The local utility of each universe does have to be constructed with care. I also have strong feelings that all utility functions are bounded. I was imagining one utility function for the multiverse, but perhaps that does not make sense. (since the collection of universes might not be a set) Perhaps the best way to model the utility function in my philosophy might be to have a separate utility function for each universe, and a simplicity exchange rate between them.

LESSWRONG
LW

LESSWRONG
LW

27

Preferences without Existence

27

27