Probability is Subjectively Objective

-18The_Compassionate_Cynic

2DSimon

6Abigail

-2DanielLC

-4venomwolfse

2Allan_Crossman

0Vladimir_Nesov

10Tim_Tyler

0Constant2

-8Barkley_Rosser

-1[anonymous]

1pdf23ds

2pdf23ds

1Cyan2

1Eliezer Yudkowsky

0Tim_Tyler

0Constant2

3Tim_Tyler

13steven

11DSimon

3orthonormal

0Constant2

0Rafe_Champion

3Tim_Tyler

0Constant2

0Vladimir_Nesov

0Tim_Tyler

0Constant2

1Tim_Tyler

0Constant2

0Tim_Tyler

-2Barkley_Rosser

0Cyan2

0Barkley_Rosser

0Cyan2

0Barkley_Rosser

0Cyan2

0Barkley_Rosser

0Sudeep_Kamath

3Tim_Tyler

0outofculture

3Yvain2

0Yvain2

3PhilGoetz

2RobinZ

2PhilGoetz

3lucidfox

14HonoreDB

2Ronny Fernandez

0Ronny Fernandez

9Ronny Fernandez

5drnickbone

0rkyeun

2Zaq

1rkyeun

0Zaq

3rkyeun

0Zaq

2rkyeun

1Zaq

-2A1987dM

9Nornagest

1A1987dM

1linkhyrule5

0Nornagest

0Lumifer

0linkhyrule5

0Lumifer

-2linkhyrule5

0SeanMCoincon

-1DragonGod

1Isaac King

New Comment

Some comments are truncated due to high volume. (⌘F to expand all)

Is not subjective objectivity the highest degree of objectivity possible for a human being?

Objective truth does exist, but people can only perceive it with their own perception filters. And, perhaps, AIs with the perception filters of their makers.

I have to decide what is truth as best I can, and may choose to assert a truth even though every one else denies it, eg Galileo. It is to my advantage to seek to make my perception filters as little distorting as possible, but I doubt I could ever achieve that completely.

-2

"for a human being" means the same thing as "subjectively". For example, attractiveness is subjective, so you could only say that someone is attractive for someone. This is as opposed to objective facts, like height. If someone is six feet tall, it doesn't matter who you ask. They might say that this person is five feet tall, but that's completely irrelevant. They're still six feet tall.
All you're saying is that subjective objectivity is the highest degree of objectivity possible while still being objective.

-4

Just like the truth, proof can only be a perspective as it falls back on whether one chooses to accept it as the truth or deny it completely. Most proofs appear extremely subjective on the back of an objective argument. Like the colours itself, my blue can be completely different from how you see your blue, but reguardless of our indifferences, what we have been told is that we see the same colour. Does that actually mean that blue is actually blue to everyone (objectively speaking ofcource) the same argument can be used with the perception of time itself, that is if time is really relevant , is this PROOF?
Knowing is not a truth. Been told something, anything doesn't mean it's true but it does not make it false either. we can only validate either self or collectively. self truth Vs collective truth. Religion is an objective truth, a belief system of something thats taught, we aren't born with a religion. Faith is subjective, it's that feeling and driving force that there is something more or a possibility of being more than what's perceived at present. In a biological way it does makes sense, after all we do have theses things called emotions and feelings, all that horrible stuff.
How do i percieve GOD? Well it's everything and nothing, that it's really indefinable. How do I relate my faith to god, well I am god. I am the creator and the destroyer the light and the darkness, well that I have the capacity to do either. I have faith in God and thats all I need, thats all anyone needs. I personally do not believe in god.
A world exists beyond the reach of our physical senses and intellect, it is deemed to be nonexistent and purely imaginary by materialist science. But quantum physics is challenging such a limited and myopic view of reality. Quantum physicists have discovered that in the quantum world, Newtonian physical laws no longer apply. What apply are the laws of consciousness.
“This denial of the nonmaterial aspect of life—its sacred participation in the mir

*there is no such thing as a probability that isn't in any mind.*

Hmm. Doesn't quantum mechanics (especially if we're forgetting about MWI) give us genuine, objective probabilities?

Probability assigned to a belief is estimated according to a goal that says "high Bayesian score is good". It has a particular optimization target, and so it doesn't run away from it. It is an "objective" fact about this optimization process that it tries to have a good Bayesian score. The probability that it produces is within its mind, and in this sense can be said to be "subjective", but it is no more subjective than my decision to take an apple from the table, that also happens in my mind and is targeted on a goal of eatin...

To answer Allan Crossman and "I never saw Jaynes spell out explicitly":

For some sixty years it has appeared to many physicists that probability plays a fundamentally different role in quantum theory than it does in statistical mechanics and analysis of measurement errors. It is a commonly heard statement that probabilities calculated within a pure state have a different character than the probabilities with which different pure states appear in a mixture, or density matrix. As Pauli put it, the former represents "Eine prinzipielle Unbestimmtheit, nicht nur Unbekanntheit". But this viewpoint leads to so many paradoxes and mysteries that we explore the consequences of the unified view, thatall probability signifies only incomplete human information.

This is from Probability In Quantum Theory (1999). Jaynes seems to be ignoring animals and AIs.

Probability isn't only used as an expression of a person's own subjective uncertainty when predicting the future. It is also used when making factual statements about the past. If a coin was flipped yesterday and came up heads 60% of the time, then it may have been a fair coin which happened to come up heads 60% of the time, or it may have been a trick, biased, coin, whose bias caused it to come up heads 60% of the time. To say that a coin is biased is to make a statement about probability. As Wikipedia explains:

In probability theory and statistics, a seq...

All true.

When E.Yudkowsky's foe agrees with something he says, you can be sure its correct ;) Of course, the answers are in the materials on ontology I posted on the everything-list months ago.

As to where an algebraic relation such as '2+3' exists, it exists in the same place as the objective value archetypes of course- it's a global feature of the Tegmark multiverse - a relation between all the possible worlds.

Probabilities, on the other hand, are not a global feature of the Tegmark multiverse, but are computed in the individual minds existing in QM bran...

*In which case, it is good that your mind does not mention people's opinions, and only thinks of the lottery balls; the ideal computation makes no mention of the lottery balls*

no mention of the people's opinions?

Allan: I would think that one of the reasons Eliezer is such a strong believer in MWI is that it *does* change the interpretation of the probability of the wavefunction from being objective back to being subjectively objective.

Barkley Rosser, I think the difference between objective Bayesians and subjective Bayesians has more to do with how they treat prior distributions than how they view asymptotic convergence.

I'm personally not an objective Bayesian by your definition -- I don't think there are stable, true probability distributions "out there." Nevertheless, I do find the asymptotic convergence theorems meaningful. In my view, asymptotic convergence gets you to the most informative distribution conditional on some state of information, but that state of information...

Re: *a statement about probability can enter into a factual claim about the causes of past events*.

Not under the view we are discussing. Did you read the referenced Probability is in the Mind page?

The idea is that uncertainty is a psychological phenomenon. It exists in the mind, not in the world.

In order to criticise Jaynes' perspective, you should first understand it. It is not clear to me that you have done that.

Jaynes' perspective on the historical behaviour of biased coins would make no mention of probability - unless he was talking about the history of the expectations of some observer with partial information about the situation. Do you see anything wrong with that?

*Reality is that which, when you stop believing in it, doesn't go away.*

This is false, of course; with sufficiently advanced technology you could build a machine that read out your mind state and caused Earth to disappear once it determined you no longer believed in Earth. Doesn't mean Earth was never real.

3

I don't believe such a machine is possible.

*Jaynes' perspective on the historical behaviour of biased coins would make no mention of probability - unless he was talking about the history of the expectations of some observer with partial information about the situation. Do you see anything wrong with that?*

I see nothing wrong with that. Similarly, if someone mentions only the atoms in my body, and never mentions me, there is nothing wrong with that. However, I am also there.

What I have pointed out is that seemingly unproblematic statements can indeed be made of the sort that I described. That Jaynes h...

A couple of comments on a very big topic.

There is a fascinating history of ideas about the objective contents of thought, starting with the Austrians Brentano and Meinong, running through analytical philosophy (Russell and Moore) and phenomenology (Husserl and Heidegger), and also through evolutionary epistemology (Popper and Munz). http://www.the-rathouse.com/EvenMoreAustrianProgram/EMAThreeAustrianStrands.html

On the Bayesian appraisal of theories, with reference to the Duhem problem, it seems that Bayes gives a good result when there is only one majo

Re: *seemingly unproblematic statements can indeed be made of the sort that I described*.

The example appears to be that of the *accursèd frequentists* - the view that Jaynes spent much of his academic life crusading against.

If such ideas *seem* unproblematic to *you*, that's fine - but be aware that others find the notion of real word probabilities to be unsupported by evidence, and contrary to Occam's razor.

*If such ideas seem unproblematic to you*

It is the example that seems on the face of it unproblematic. I am open to either (a) a demonstration that it is compatible with subjectivism[*], or (b) a demonstration that it is problematic. I am open to either one. Or to something else. In any case, I don't adhere to frequentism.

[*] (I made no firm claim that it is not compatible with subjectivism - you are the one who rejected the compatibility - my own purpose was only to raise the question since it seems on the face of it hard to square with subjectivism, not to answer the question definitively.)

steven: *This is false, of course; with sufficiently advanced technology you could build a machine that read out your mind state and caused Earth to disappear once it determined you no longer believed in Earth. Doesn't mean Earth was never real.*

There is no clear separation between the mind and the territory. The structure of the mind is instrumental to optimization process. When you change the beliefs, change the state of the mind, you are in fact performing an action on the territory, that is instrumental to the goal. Establishing specific state (process) ...

I *thought* we had agreed that the historical behaviour of coins was "compatible with subjectivism":

Jaynes' perspective on the historical behaviour of biased coins would make no mention of probability - unless he was talking about the history of the expectations of some observer with partial information about the situation.

An observer with one set of partial information might have predicted a coin would come up heads 50% of the time. An observer with another set of partial information might have predicted a coin would come up heads 60% of the ti...

*With complete information (and a big computer) an observer would know which way the coin would land - and would find probabilities irrelevant.*

But this is true of most everyday observations. We observe events on a level far removed from the subatomic level. With complete information and infinite computing power an observer would would find all or virtually all ordinary human-level observations irrelevant. But irrelevancy to such an observer is not the same thing as non-reality. For example, the existence of elephants would be irrelevant to an observer who h...

Re: *Do you think it follows that elephants do not exist?*

Elephants are not properties of physics any more than probabilities are.

The concept of an elephant is subjective - as are all concepts.

The atoms composing an elephant are real enough.

AFAICS, nobody ever claimed that probabilites "do not exist". The idea is that uncertainty is a *psychological* phenomenon, not that it is *non-existent*.

I hope that clarifies things.

*Elephants are not properties of physics any more than probabilities are. The concept of an elephant is subjective - as are all concepts.*

If you are indeed agreeing with the parallel I have set up between probability and elephants and if this is not just your own personal view, then perhaps the subjectivist theory of probability should more properly be called the subjectivist theory of pretty much everything that populates our familiar world. Anyway, I think I can agree that probability is as subjective and as psychological and as non-physical and as existing in the mind and not in the world as an elephant or, say, an exploding nuclear bomb - another item that populates our familiar world.

We don't have people claiming that elephants are a concept in fundamental physics. However, people *do* claim that about probability. Thus the focus on probability and not on elephants.

Cyan,

OK, I grant your point. However, assuming that there is some "subjectively real" probability distribution that the Bayes' Theorem process will converge is a mighty strong assumption.

Barkley Rosser, it's a strong assumption in principle, but in practice, humans seem to be pretty good at obtaining enough information to put in the model such that the posterior does in fact converge to *some* point in the parameter space.

Cyan,

Why should there be convergence to some such point when there is no underlying "true" distribution, either subjective or objective? Are you counting on herding by people? It is useful to keep in mind the conditions under which even in classical stats, Bayes' Theorem does not hold, for example when the underlying distribution is not continuous or if it is infinite dimensional. In the former case convergence can be to a cycle of bouncing back and forth between the various disconnected portions of the distribution. This can happen, presumably in a looser purely subjective world, with even a multi-modal distribution.

Barkley Rosser, what I have in mind is a reality which in principle predictable given enough information. So there is a "true" distribution -- it's conditional on information which specifies the state of the world exactly, so it's a delta function at whatever the observables actually turn out to be. Now, there exists unbounded sequences of bits which don't settle down to any particular relative frequency over the long run, and likewise, there is no guarantee that any particular sequence of observed data will lead to my posterior distribution gett...

Barkley Rosser, there definitely is something a little hinky going on in those infinite dimensional model spaces. I don't have the background in measure theory to really grok that stuff, so I just thank my lucky stars that other people have proven the consistency of Dirichlet process mixture models and Gaussian process models.

Gotta have that continuous support too, which is the real key to converging on a cycle rather than a point.

In the fuzzier world of not a definite for-real underlying distribution, I note that multiple equilibria or basins in dynamical systems can give the multi-modality that within a herding framework can lead to some sort of cycle in bouncing back and forth between the dominant states.

"Probability exists in your mind: if you're ignorant of a phenomenon, that's an attribute of you, not an attribute of the phenomenon."

Eliezer, a small point here: if QM is true, then the universe and phenomena within the same are inherently probabilistic, are they not?

Great post as usual.

Re: *if QM is true, then the universe and phenomena within the same are inherently probabilistic*

Not really - e.g. see:

Q13 Is many-worlds a deterministic theory? Yes, many-worlds is a deterministic theory [...]

Beside the point, but you can calculate arbitrary digits of pi with the formula explained in this article.

I was one of the people who suggested the term h-right before. I'm not great with mathematical logic, and I followed the proof only with difficulty, but I think I understand it and I think my objections remain. I think Eliezer has a brilliant theory of morality and that it accords with all my personal beliefs, but I still don't understand where it stops being relativist.

I agree that some human assumptions like induction and Occam's Razor have to be used partly as their own justification. But an ultimate justification of a belief has to include a reason for...

...yeah, this was supposed to go in the new article, and I was just checking something in this one and accidentally posted it here. Please ignore *embarrassed*

"Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick

I have to comment on the irony of this quote. Philip K. Dick's novels are almost all extended riffs on the idea that there is no reality; or that reality is unknowable or irrelevant (eg. The Man in the High Castle, Flow My Tears the Policemen Said, Do Androids Dream of Electric Sheep? (Blade Runner), We Can Remember It for You Wholesale (Total Recall)). And Philip K. Dick was unable to distinguish reality from fantasy in everyday life.

2

Perhaps that's why he had to invent better tests.

But it also makes a deep philosophical point as well, which I never saw Jaynes spell out explicitly, but I think he would have approved: there is no such thing as a probability that isn't in any mind. Any mind that takes in evidence and outputs probability estimates of the next event, remember, can be viewed as a prior - so there is no probability without priors/minds.

Is this why you believe so strongly in Many-Worlds? To avoid mind-free, objective, quantum-mechanical probabilities?

I felt all the way through this post like it was confronting a difficu...

When you judge those alternate minds, you'll do so using your own mind - your own beliefs about the universe - your own posterior that came out of your own prior, your own posterior probability assignments P(X|A,B,C,...,Q1).

Hehe, "your own posterior".

[strangles her inner 12-year-old]

I am almost convinced, honestly. I was leaning towards a frequentest view, but I'm realizing now -- as pointed out here by a fellow community member -- that some of my statements are similar if not identical to the conclusion here:

Jaynes certainly believed very firmly that probability was in the mind... But Jaynes also didn't think that this implied a license to make up whatever priors you liked. There was only one correct prior distribution to use, given your state of partial information at the start of the problem.

is pretty similar to :

...Bayesian re

Now, can't I be a philosophical frequentest and a subjective bayesian? Just because probability theory models subjective beliefs does not mean that it doesn't model frequencies; in fact, if some body told me that bayes doesn't model frequencies I'm pretty sure I could prove them wrong much more easily than someone who said that probabilities don't model degrees of belief.

But there is no contradiction in saying that the komolgorov probability function models both degrees of beliefs and actual frequencies.

(edit)

In fact it seems to me that komolgorov plai...

5

This is pretty close to my own position... Probability is strictly a mathematical concept (Kolmogorov axioms). Real-world probability is anything that can be successfully modelled by the Kolmogorov axioms. This applies to both betting probabilities (violate the axioms and you get Dutch-booked) and relative frequencies.
I'm a little bit puzzled by Eliezer's view that probability is purely Bayesian, as he also believes in a "Big World", and the relative frequency approach works extremely well in a Big World (as long as it is an infinitely-big world). Chancy events really do get repeated infinitely many times, the repetitions really are independent (because of large separation, and locality of physics), and the relative frequencies really are defined and really do converge to exactly what QM says the probabilities are. All works fine.
Also, there is a formal isomorphism between decoherent branches of a wave function (as applied to a single causal region) and spatially-separated causal regions in a multiverse. So you can, if you like, consider a single space time multiverse with an intuitive interpretation (other universes are just really far away) and forget about all the splitting. Bousso and Susskind have a nice recent paper about this: http://arxiv.org/abs/1105.3796

"Perform the experiment a hundred times, and—" Okay, let's talk about the ten trillionth digit of pi, then. Single-shot problem, no "long run" you can measure.

And there goes my belief in any kind of probability as a phenomenon. I don't know what the ten trillionth digit of pi is, but I know the algorithm which generates it, and it never involves a die roll or coin flip of any kind. And if the universe is to be lawful, it doesn't roll dice either. There is no probability. To say there was is to say the ten trillionth digit of pi might somehow have come out differently. And that would be unlawful.

2

This is silly. To say that there is some probability in the universe is not to say that everything has randomness to it. People arguing that there is intrinsic probability in physics don't argue that this intrinsic probability finds its way into the trillionth digit of pi.
Many Physicists: If I fire a single electron at two slits, with a detector placed immediately after one of the slits, then I detect the electron half the time. Furthermore, leading physics indicates that no ammount of information will ever allow me to accurately predict which trials will result in a detected electron, I can determine a 50/50 chance for detection/non-detection and that's the limit of predictability. Thus it's safe to say that the 50/50 is a property of the experimental set-up, and not a property of how much I know about the setup.
Pretty Much Zero Physicists: The above indicates that the trillionth digit of pi is in a superposition until we calculate it, at which point it collapses to a single value.

1

Dr. Many the Physicist would be wrong about the electron too. The electron goes both ways, every time. There's no chance involved there either.
But you're right, it is not the ten trillionth digit of pi that proves it.

0

The Many Physicists description never talked about the electron only going one way. It talked about detecting the electron. There's no metaphysics there, only experiment. Set up a two-slit configuration and put a detector at one slit, and you see it firing half the time. You may say that the electron goes both ways every time, but we still only have the detector firing half the time. We also cannot predict which half of the trials will have the detector firing and which won't. And everything we understand about particle physics indicates that both the 1/2 and the trial-by-trial unpredictability is NOT coming from ignorance of hidden properties or variables but from the fundamental way the universe works.

3

No, I see it firing both ways every time. In one world, I see it going left, and in another I see it going right. But because these very different states of my brain involve a great many particles in different places, the interactions between them are vanishingly nonexistent and my two otherworld brains don't share the same thought. I am not aware of my other self who has seen the particle go the other way.
We have both detectors firing every time in the world which corresponds to the particle's path. And since that creates a macroscopic divergence, the one detector doesn't send an interference signal to the other world.
We can predict it will go both ways each time, and divide the world in twain along its amplitude thickness, and that in each world we will observe the way it went in that world. If we are clever about it, we can arrange to have all particles end in the same place when we are done, and merge those worlds back together, creating an interference pattern which we can detect to demonstrate that the particle went both ways. This is problematic because entanglement is contagious, and as soon as something macroscopic becomes affected putting Humpty Dumpty back together again becomes prohibitive. Then the interference pattern vanishes and we're left with divergent worlds, each seeing only the way it went on their side, and an other side which always saw it go the other way, with neither of them communicating to each other.
Correct. There are no hidden variables. It goes both ways every time. The dice are not invisible as they roll. There are instead no dice.

0

You've only moved the problem down one step.
Five years ago I sat in a lab with a beam-spitter and a single-photon multiplier tube. I watched as the SPMT clicked half the time and didn't click half the time, with no way to predict which I would observe. You're claiming that the tube clicked every time, and the the part of me that noticed one half is very disconnected from the part of me that noticed the other half. The problem is that this still doesn't allow me to postdict which of the two halves the part of me that is typing this should have in his memory right now.
Take the me sitting here right now, with the memory of the specific half of the clicks he has right now. As far as we understand physics, he can't postdict which memory that should have been. Even in your model, he can postdict that there will be many branches of him with each possible memory, but he can't postdict which of those branches he'll be - only the probability of him being any one of the branches.

2

Moving the problem down one step puts it at the bottom.
One half of you should have one, and the other half should have the other. You should be aware intellectually that it is only the disconnect between your two halves' brains not superimposing which prevents you from having both experiences in a singular person, and know that it is your physical entanglement with the fired particle which went both ways that is the cause. There's nothing to post-dict. The phenomenon is not merely explained, but explained away. The particle split, on one side there is a you that saw it split right, on one side there is a you that saw it split left, and both of you are aware of this fact, and aware that the other you exists on the other side seeing the other result, because the particle always goes both ways and always makes each of you. There is no more to explain. You are in all branches, and it is not mysterious that each of you in each branch sees its branch and not the others. And unless some particularly striking consequence happened, all of them are writing messages similar to this, and getting replies similar to this.

1

The issue is not want of an explanation for the phenomenon, away or otherwise. We have an explanation of the phenomenon, in fact we have several. That's not the issue. What I'm talking about here is the inherent, not-a-result-of-my-limited-knowledge probabilities that are a part of all explanations of the phenomenon.
Past me apparently insisted on trying to explain this in terminology that works well in collapse or pilot-wave models, but not in many-worlds models. Sorry about that. To try and clear this up, let me go through a "guess the beam-splitter result" game in many-worlds terminology and compare that to a "guess the trillionth digit of pi" game in the same terminology.
Aside: Technically it's the amplitudes that split in many-worlds models, and somehow these amplitudes are multiplied by their complex conjugates to get you answers to questions about guessing games (no model has an explanation for that part). As is common around these parts, I'm going to ignore this and talk as if it's the probabilities themselves that split. I guess nobody likes writing "square root" all the time.
Set up a 50/50 beam-splitter. Put a detector in one path and block the other. Write your choice of "Detected" or "Not Detected" on a piece of paper. Now fire a single photon. In Everett-speak, half of the yous end up in branches where the photon's path matches your guess while half of the yous don't. The 50/50 nature of this split remains even if you know the exact quantum state of the photon beforehand. Furthermore, the branching of yous that try to use all your physics knowledge to predict their observations have no larger a proportion of success than the branching of yous that make their predictions by just flipping a coin, always guessing Fire, or employing literally any other strategy that generates valid guesses. The 50/50 value of this branching process is completely decoupled from your predictions, no matter what information you use to make those predictions.
Compare this

"Reality is that which, when you stop believing in it, doesn't go away."

By that definition, stuff like the value of the US dollar aren't real.

9

Whether you personally believe it or not, most gas stations in the US will be happy to exchange $1.50 or thereabouts for a cup of sludgy, lukewarm coffee. That may not be grounded on par with Newton's laws of motion, but it still seems like a step up from, say, body thetans.

1

That's because the gas station owner still believes in the value of the dollar. But if all of you stopped believing in it...

1

Right, but if I had an effect that depended on massed belief, I'd be pretty sure that existed too.
Say, Discworld-style gods.
(Things that depend on individual belief generally have a low enough prior that I'm probably dreaming/hallucinating, and also I can't tell myself to stop believing in things so whatever it is I think I'm doing it's not "not believing in X".)

0

Sure. When we're looking at socially constructed stuff, though, I'm not sure it makes sense to treat its reality as discrete rather than continuous.
It's tempting to use money as a measure of social reality, actually, since it's valuable directly in proportion to people's belief in its value. Unfortunately it's hard to put a dollar value on, say, traffic laws, which share the same property (a point driven home for me the first time I encountered rush-hour traffic in Manila).

0

Of course.
Actually, all value (which exists solely inside minds) is not real.

0

...
Things that don't exist can't have effects.
Religion is real. God isn't, but the belief-in-god is and has effects.
Value - a shared pattern between brains - is real and has effects.

0

Look at the quote in the grandparent. For the purpose of this thread I'm using the word "real" in the sense of "existing outside and independently of your mind". By that approach beliefs are not real and sentences "X exists" and "X is real" have very different meanings.

-2

If you stop believing in religion, that does not stop fundamentalists from existing.
If you stop thinking money has value, the economy will not collapse.
Even if you don't trust your own brain, a shared meme can still be "real."

Somewhere out in mind design space, there's a mind with any possible prior; but that doesn't mean that you'll say, "All priors are created equal."

The corrected phrase may be: "All *unentangled* priors are created equal."

I think this is the reason why a distinction between subjective and objective probability is needed.

If we figure out how to build GAI, we could build several with different priors, release them into the universe, and see which ones do better. If we give them all the same metric to optimize, they will all agree on which of them did better, thus determining one prior that is the best one to have for this universe.

Followup to: Probability is in the MindThere are two kinds of Bayesians, allegedly. Subjective Bayesians believe that "probabilities" are degrees of uncertainty existing in our minds; if you are uncertain about a phenomenon, that is a fact about your state of mind, not a property of the phenomenon itself; probability theory constrains the logical coherence of uncertain beliefs. Then there are objective Bayesians, who... I'm not quite sure what it means to be an "objective Bayesian"; there are multiple definitions out there. As best I can tell, an "objective Bayesian" is anyone who uses Bayesian methods and isn't a subjective Bayesian.

If I recall correctly, E. T. Jaynes, master of the art, once described himself as a subjective-objective Bayesian. Jaynes certainly believed very firmly that probability was in the mind; Jaynes was the one who coined the term Mind Projection Fallacy. But Jaynes also didn't think that this implied a license to make up whatever priors you liked. There was only one

correctprior distribution to use, given your state of partial information at the start of the problem.How can something be in the mind, yet still be objective?

It appears to me that a good deal of philosophical maturity consists in being able to keep separate track of nearby concepts, without mixing them up.

For example, to understand evolutionary psychology, you have to keep separate track of the psychological purpose of an act, and the evolutionary pseudo-purposes of the adaptations that execute as the psychology; this is a common failure of newcomers to evolutionary psychology, who read, misunderstand, and thereafter say, "You think you love your children, but you're just trying to maximize your fitness!"

What is it, exactly, that the terms "subjective" and "objective", mean? Let's say that I hand you a sock. Is it a subjective or an objective sock? You believe that 2 + 3 = 5. Is

your beliefsubjective or objective? What about two plus threeactuallyequaling five—is that subjective or objective? What about a specific act of adding two apples and three apples and getting five apples?I don't intend to confuse you in shrouds of words; but I do mean to point out that, while you may feel that you know very well what is "subjective" or "objective", you might find that you have a bit of trouble saying out loud what those words mean.

Suppose there's a calculator that computes "2 + 3 = 5". We punch in "2", then "+", then "3", and lo and behold, we see "5" flash on the screen. We accept this as

evidencethat 2 + 3 = 5, but we wouldn't say that the calculator's physical outputdefinesthe answer to the question 2 + 3 = ?. A cosmic ray could strike a transistor, which might give us misleading evidence and cause us to believe that 2 + 3 = 6, but it wouldn't affect theactualsum of 2 + 3.Which proposition is common-sensically true, but philosophically interesting: while we can easily point to the physical location of a symbol on a calculator screen, or observe the result of putting two apples on a table followed by another three apples, it is rather harder to track down the whereabouts of 2 + 3 = 5. (Did you look in the garage?)

But let us leave aside the question of

wherethe fact 2 + 3 = 5 is located—in the universe, or somewhere else—and consider the assertion that the proposition is "objective". If a cosmic ray strikes a calculator and makes it output "6" in response to the query "2 + 3 = ?", and you add two apples to a table followed by three apples, then you'll still see five apples on the table. If you do the calculation in your own head, expending the necessary computing power—we assume that 2 + 3 is a very difficult sum to compute, so that the answer is not immediately obvious to you—then you'll get the answer "5". So the cosmic ray strike didn't change anything.And similarly—exactly similarly—what if a cosmic ray strikes a neuron inside your brain, causing you to compute "2 + 3 = 7"? Then, adding two apples to three apples, you will expect to see seven apples, but instead you will be surprised to see five apples.

If instead we found that no one was ever mistaken about addition problems, and that, moreover, you could change the answer by an act of will, then we might be tempted to call addition "subjective" rather than "objective". I am not saying that this is

everythingpeople mean by "subjective" and "objective", just pointing to one aspect of the concept. One might summarize this aspect thus: "If you can change something by thinking differently, it's subjective; if you can't change it by anything you do strictly inside your head, it's objective."Mind is not magic. Every act of reasoning that we human beings carry out, is

computed withinsome particular human brain. But not every computation isaboutthe state of a human brain. Not every thought that you think isaboutsomething that can be changed by thinking. Herein lies the opportunity for confusion-of-levels. The quotation is not the referent. If you are going to consider thoughts as referential at all—if not, I'd like you to explain the mysterious correlation between my thought "2 + 3 = 5" and the observed behavior of apples on tables—then, while the quoted thoughts will always change with thoughts, the referentsmay or may notbe entities that change with changing human thoughts.The calculator computes "What is 2 + 3?", not "What does this calculator compute as the result of 2 + 3?" The answer to the former question is 5, but if the calculator were to ask the latter question instead, the result could self-consistently be anything at all! If the calculator returned 42, then indeed, "What does this calculator compute as the result of 2 + 3?" would in fact be 42.

So just because a computation takes place inside your brain, does not mean that the computation

explicitly mentionsyour brain, that it has your brain as areferent,any more than the calculator mentions the calculator. The calculator does not attempt to contain a representation of itself, only of numbers.Indeed, in the most straightforward implementation, the calculator that asks "What does this calculator compute as the answer to the query 2 + 3 = ?" will

neverreturn a result, just simulate itself simulating itself until it runs out of memory.But if you punch the keys "2", "+", and "3", and the calculator proceeds to compute "What do I output when someone punches '2 + 3'?", the resulting computation does have one interesting characteristic: the

referentof the computation is highly subjective, since it depends on the computation, and can be made to be anything just by changing the computation.Is probability, then, subjective or objective?

Well, probability is computed within human brains or other calculators. A probability is a state of partial information that is possessed by you; if you flip a coin and press it to your arm, the coin is showing heads or tails, but you assign the probability 1/2 until you reveal it. A friend, who got a tiny but not fully informative peek, might assign a probability of 0.6.

So can you make the probability of winning the lottery be anything you like?

Forget about many-worlds for the moment—you should almost always be able to forget about many-worlds—and pretend that you're living in a single Small World where the lottery has only a single outcome. You will nonetheless have a need to call upon probability. Or if you prefer, we can discuss the ten trillionth decimal digit of pi, which I believe is not yet known. (If you are foolish enough to refuse to assign a probability distribution to this entity, you might pass up an excellent bet, like betting $1 to win $1000 that the digit is not 4.) Your uncertainty is a state of your mind, of partial information that you possess. Someone else might have different information, complete or partial. And the entity itself will only ever take on a single value.

So can you make the probability of winning the lottery, or the probability of the ten trillionth decimal digit of pi equaling 4, be anything you like?

You might be tempted to reply: "Well, since I

currentlythink the probability of winning the lottery is one in a hundred million, then obviously, I willcurrentlyexpect that assigning any other probability than this to the lottery, will decrease my expected log-score—or if you prefer a decision-theoretic formulation, I will expect this modification to myself to decrease expected utility. So, obviously, I will not choose to modify my probability distribution. It wouldn't be reflectively coherent."So reflective coherency is the goal, is it? Too bad you weren't born with a prior that assigned probability 0.9 to winning the lottery! Then, by exactly the same line of argument, you wouldn't want to assign any probability except 0.9 to winning the lottery. And you would still be reflectively coherent. And you would have a 90% probability of winning millions of dollars! Hooray!

"No, then I would

thinkI had a 90% probability of winning the lottery, butactually,the probability would only be one in a hundred million."Well, of course

youwould be expected to say that. And if you'd been born with a prior that assigned 90% probability to your winning the lottery, you'd consider an alleged probability of 10^-8, and say, "No, then I wouldthinkI had almost no probability of winning the lottery, butactually,the probability would be 0.9.""Yeah? Then just modify your probability distribution, and buy a lottery ticket, and then wait and see what happens."

What happens? Either the ticket will win, or it won't. That's what will happen. We won't get to see that some particular probability was, in fact, the exactly right probability to assign.

"Perform the experiment a hundred times, and—"

Okay, let's talk about the ten trillionth digit of pi, then. Single-shot problem, no "long run" you can measure.

Probability is subjectively objective: Probability exists in your mind: if you're ignorant of a phenomenon, that's an attribute of you, not an attribute of the phenomenon. Yet it will seem to you that you can't change probabilities by wishing.

You could make yourself compute something

else,perhaps,rather thanprobability. You could compute "What do I say is the probability?" (answer: anything you say) or "What do I wish were the probability?" (answer: whatever you wish) but these things are not theprobability,which is subjectively objective.The thing about subjectively objective quantities is that they

really doseem objective to you. You don't look them over and say, "Oh, well, of course I don't want to modify my own probability estimate, because no one can just modify their probability estimate; but if I'd been born with a different prior I'd be saying something different, and I wouldn't want to modify that either; and so none of us is superior to anyone else." That's the way a subjectivelysubjectivequantity would seem.No, it will seem to you that, if the lottery sells a hundred million tickets, and you don't get a peek at the results, then the probability of a ticket winning,

isone in a hundred million. And that you could be born with different priors but that wouldn't give you any better odds. And if there's someone next to you saying the same thing abouttheir90% probability estimate, you'll just shrug and say, "Good luck with that." You won't expect them towin.Probability is subjectively

reallyobjective, not just subjectivelysort ofobjective.Jaynes used to recommend that no one ever write out an unconditional probability: That you never, ever write simply P(A), but always write P(A|I), where I is your prior information. I'll use Q instead of I, for ease of reading, but Jaynes used I. Similarly, one would not write P(A|B) for the posterior probability of A given that we learn B, but rather P(A|B,Q), the probability of A given that we learn B and had background information Q.

This is good advice in a purely pragmatic sense, when you see how many false "paradoxes" are generated by accidentally using different prior information in different places.

But it also makes a deep philosophical point as well, which I never saw Jaynes spell out explicitly, but I think he would have approved:

there is no such thing as a probability that isn't in any mind.Any mind that takes in evidence and outputs probability estimates of the next event, remember, can be viewed as a prior—so there is no probability without priors/minds.You can't unwind the Q. You can't ask "What is the

unconditionalprobability of our background information being true, P(Q)?" To make that estimate, you would still needsomekind of prior. No way to unwind back to an ideal ghost of perfect emptiness...You might argue that you and the lottery-ticket buyer do not really have a disagreement about

probability.You say that the probability of the ticket winning the lottery is one in a hundred million given your prior, P(W|Q1) = 10^-8. The other fellow says the probability of the ticket winning given his prior is P(W|Q2) = 0.9. Every timeyousay "The probability of X is Y", you really mean, "P(X|Q1) = Y". And whenhesays, "No, the probability of X is Z", he reallymeans,"P(X|Q2) = Z".Now you might, if you traced out his mathematical calculations, agree that, indeed, the conditional probability of the ticket winning, given his weird prior is 0.9. But you wouldn't agree that "the probability of the ticket winning" is 0.9. Just as he wouldn't agree that "the probability of the ticket winning" is 10^-8.

Even if the two of you refer to different mathematical calculations when you say the word "probability",

youdon't think that puts you on equal ground, neither of you being better than the other. And neither does he, of course.So you see that, subjectively, probability really

doesfeel objective—even after you have subjectively taken all apparent subjectivity into account.And this is not mistaken, because, by golly, the probability of winning the lottery really

is10^-8, not 0.9. It's not as if you're doing your probability calculationwrong,after all. If you weren't worried about being fair or about justifying yourself to philosophers, if you only wanted to get the correct answer, your betting odds would be 10^-8.Somewhere out in mind design space, there's a mind with any possible prior; but that doesn't mean that you'll say, "All priors are created equal."

When you judge those alternate minds, you'll do so using your own mind—your own beliefs about the universe—your own posterior that came out of your own prior, your own posterior probability assignments P(X|A,B,C,...,Q1). But there's nothing wrong with that. It's not like you could judge using something other than yourself. It's not like you could have a probability assignment without any prior, a degree of uncertainty that isn't in any mind.

And so, when all that is said and done, it still seems like the probability of winning the lottery really

is10^-8, not 0.9. No matter what other minds in design space say differently.Which shouldn't be surprising. When you compute probabilities, you're thinking about lottery balls, not thinking about brains or mind designs or other people with different priors. Your probability computation makes no mention of that, any more than it explicitly represents itself. Your goal, after all, is to win, not to be fair. So of course probability will

seemto be independent of what other minds might think of it.Okay, but... you

stillcan't win the lottery by assigning a higher probability to winning.If you like, we could regard probability as an idealized computation, just like 2 + 2 = 4 seems to be independent of any particular error-prone calculator that computes it; and you could regard your mind as trying to approximate this ideal computation. In which case, it is good that your mind does not mention people's opinions, and only thinks of the lottery balls; the ideal computation makes no mention of people's opinions, and we are trying to reflect this ideal as accurately as possible...

But what you will calculate is the "ideal calculation" to plug into your betting odds, will depend on your prior, even though the calculation won't have an explicit dependency on "your prior". Someone who thought the universe was anti-Occamian, would advocate an anti-Occamian calculation, regardless of whether or not anyone thought the universe was anti-Occamian.

Your calculations get checked against reality, in a probabilistic way; you either win the lottery or not. But interpreting these results, is done with your prior; once again there is no probability that isn't in any mind.

I am not trying to argue that you can win the lottery by wishing, of course. Rather, I am trying to inculcate the ability to

distinguish between levels.When you think about the ontological nature of probability, and perform reductionism on it—when you try to explain how "probability" fits into a universe in which states of mind do not exist

fundamentally—then you find that probability is computed within a brain; and you find that other possible minds could perform mostly-analogous operations with different priors and arrive at different answers.But, when you consider probability

as probability,think about thereferentinstead of the thought process—which thinking you will do in your own thoughts, which are physical processes—then you will conclude that the vast majority of possible priors areprobably wrong. (You will also be able to conceive of priors which are, in fact, better than yours, because they assign more probability to the actual outcome; you just won't know in advance which alternative prior is the truly better one.)If you again swap your goggles to think about how probability is implemented in the brain, the seeming objectivity of probability is the way the probability algorithm feels from inside; so it's no

mysterythat, considering probability as probability, you feel that it's not subject to your whims. That's just what the probability-computation would be expected to say, since the computation doesn't represent any dependency on your whims.But when you swap out those goggles and go back to thinking about probabilities, then, by golly, your algorithm seems to be

rightin computing that probability is not subject to your whims. Youcan'twin the lottery just by changing your beliefs about it. And if that is the way you would be expected to feel, then so what? The feeling has been explained, not explained away; it is not amerefeeling. Just because a calculation is implemented in your brain, doesn't mean it'swrong,after all.Your "probability that the ten trillionth decimal digit of pi is 4", is an attribute of yourself, and exists in your mind; the real digit is either 4 or not. And if you could change your belief about the probability by editing your brain, you wouldn't expect that to change the probability.

Therefore I say of probability that it is "subjectively objective".

Part of

The Metaethics SequenceNext post: "Whither Moral Progress?"

Previous post: "Rebelling Within Nature"