# 22

Let's say Omega turns up and sets you a puzzle, since this seems to be what Omega does in his spare time.  He has with him an opaque jar, which he says contains some solid-colored beads, and he's going to draw one bead out of the jar.  He would like to know what your probability is that the bead will be red.

Well, now there is an interesting question.  We'll bypass the novice mistake of calling it .5, of course; just because the options are binary (red or non-red) doesn't make them equally likely.  It's not like you have any information.  Assuming you don't think Omega is out to deliberately screw with you, you could say that the probability is .083 based on the fact that "red" is one of twelve basic color words in English.  (If he had asked for the probability that the bead would be lilac, you'd be in a bit more trouble.)  If you were obliged to make a bet that the bead is red, you would probably take the most conservative bet available (even if you're still assuming Omega isn't deliberately screwing with you), but .083 sounds okay.

So that's one kind of probability: the bead jar guess.  It has a basis, but it's a terribly flimsy one, and guessing right (or wrong) doesn't help much to confirm or disconfirm the guess.  Even if Omega had asked about the bead being lilac, and you'd dutifully given a tiny probability, it would not have surprised you to see a lilac bead emerge from the jar.

A non-bead-jar-guess probability yields surprise when it turns out to be true even if it's just the same size.  Say your probability for lilac was .003.  That's tiny.  If you had a probability of .003 that it would rain on a particular day, you would be right to be astonished if you turned out to need the umbrella you left at home.

I think more of our beliefs are bead jar guesses than we realize, but because of assorted insidious psychological tendencies, we don't recognize that and we hold onto them tighter than baseless suppositions deserve.

# 22

New Comment
Some comments are truncated due to high volume. Change truncation settings
[-]jimmy130

"We'll bypass the novice mistake of calling it .5, of course; just because the options are binary (red or non-red) doesn't make them equally likely. It's not like you have any information."

Well, if you truly had no information, 0.5 would be the correct (entropy maximizing given constraints) bet. If you have no information you can call it "A or !A" or "!B or B" and it sounds the same- you can't say one is more likely.

By assigning a different probability, you're saying that you have information that makes the word "red" means something to you, and that it's less likely than half (say because there are 11 other "colors").

Likewise, if I say how likely is A and how likely is !A? You have to say 0.5. If A turns out to be "I'm gonna win the lottery tomorrow" then you can update and P goes to near zero. You didn't screw up though, since It could have just as easily been "I won't win the lottery tomorrow". If you don't think that it's just as likely, then that is information.

When you hear people saying "winning the lottery is 50/50 because either you win or you don't", their error isn't that they "naively" predict 0.5 in total absence of information. Their problem is that they don't update on the information that they do have.

2Alicorn
Well, I suppose you do have information inasmuch as you know what colors are. But if your probability for red is .5, on the basis of knowing that it's a color alone, then you have to have the same probabilities for blue and yellow and green and brown and so forth if Omega asks for those too, and you can be Dutch booked like crazy.
2[anonymous]
If Omega asks "What is p(red)?" then I may well consider that I have no information and reply 0.5. If Omega then asks me "What are p(blue) and p(yellow)?" then I have new information. I would update p(red), p(blue) and p(yellow) to new numbers. By induction I would probably assign each a somewhat lower probability than 0.33 by this stage since p(jar contains all basic colors) has increased. The most important thing is that I would never have exclusive probabilities that simultaniously sum to greater than 1. I may, however, declare (or even bet on) a probability that I update downwards when given new information. I may end up in a situation where all the bets I have laid sequentially have an expected loss when considered together. This is unfortunate but does not indicate an error of judgement. It simply suggests that at the time of my bet on red I did not expect the new bets or the information contained therein. In later bets I reject consistency bias.
1Alicorn
If you can be Dutch booked about probabilities asked in close sequence, where you gain no new information except that a question was asked, I'd think that reflects a considerable failure of rationality. There are grounds to reject "temporal Dutch book arguments", but this isn't one; the time and the "new information" should both be negligible. To put it differently, if you have no information about what beads are in the jar, then you have even less information about why Omega wants to know your probabilities for the sorts of beads in the jar. Omega is a weird dude. Omega asking you a question does not mean what it means when a human asks you the same question.
2[anonymous]
0saturn
And it's always .5, I hope.
4Unknowns
Your probability of updating downwards should be (more or less; not exactly) equal to one minus your original probability, i.e. if your original probability is .25, your probability of updating downwards should be around .75. This is obvious, since if there is a one in four chance that the thing is so, there is a three out of four chance that you will find out that it is not so, when you find out whether it is so or not. Conservation of expected evidence doesn't mean that the chance of updating upwards is equal to the chance of updating downwards. It also takes into account the quantity of the change; i.e. my probability is .25, and I update upwards, I will have to update three times as much as if I had updated downwards.
1saturn
You're right. Thanks.
0sparkles
What if you know jar A is 80% red and jar B is 0% red, and you know you're looking at one of them, and your confidence that it's A is 0.625? Then you have probability 0.5 that a bead chosen from the jar in front of you is red, but will update upwards with probability 0.625 if you're given the information of which jar you're looking at.
0Unknowns
My comment assigns to a probability to updating upwards or downwards in a generic way when new information is given; your comment calculates based on "if you're given the information of which jar you're looking at", which is more concrete. You could also be given other information which would make it more likely you're looking at B.
No, it's not. (You either win the lottery, or you don't.)
2Mike Bishop
Excuse me for making such a minor point, but I don't think we have to give the same probability for each color. We have to guess at Omega's motivation before we can guess at the distribution of bead colors in the jar. Do we have previous knowledge of Omegas? How about Omegas bearing bead filled jars?
1Alicorn
I was assuming that you have never met an Omega, much less one bearing a bead jar, and that you know all the standard facts about Omega (e.g. what he says is true, etc.)
0mitechka
I think I would agree partially with both of you. If I assume that there is no information at all .5 is a good choice. Once a bead of any color is pulled out, I can start making guesses on a potential number of beads in the jar from the relative volumes of the jar and the bead, so if I know that there is a finite number of potential colors, I might take a guess as to what the probability of any particular color distribution is. Once a red bead is pulled, I might adjust probability that Omega is not screwing with me etc.

I think more of our beliefs are bead jar guesses than we realize, but because of assorted insidious psychological tendencies, we don't recognize that and we hold onto them tighter than baseless suppositions deserve. [said Alicorn in the original]

If someone wants to do the work of linking the fairly abstract discussion here to how we think about making decisions in the real world, I think we would all benefit greatly.

We're to assume here that Omega, being a strange sort of deity, has selected the question "Is the bead red?" via some process that has no expected correlation whatsoever to the actual color. Correct?

4Alicorn
Except inasmuch as red is in fact a color (not a euphemism for communist beads or something), yes, that's right.
4Emile
That's an important information, and one that wasn't included in the specification of the problem - hence some people in the comments arguing that 0.5 may not be that stupid.
0timtyler
Would you increase or decrease the probability on the basis of this information?
0Emile
Decrease - wouldn't you?
1timtyler
0Alicorn
Okay, I'll bite - did anybody really think Omega might have been asking for the probability that the first bead would be a Communist?
3MrHen
No, but if the first bead was communist I sure am going to start thinking the rest of the beads were communist. Those silly communes always stick together. With more seriousness, I am guessing Emile was talking about the revelation that the contents of the jar were restricted by "color" as in one of twelve colors.
0Emile
I wasn't talking about communism, I don't know where you picked that up. I was just saying that the assumption "Omega's choice of question is uncorrelated to the actual bead color" is missing, and should be explicitly stated. Otherwise, it's reasonable to assign non-null probabilities to the propositions "There are beads of only N colors, and red is one of them", for various values of N (2? 12? Whatever) Cameron Taylor makes the same point in another comment.
-2[anonymous]
That's a fairly significant exception.

I've just started reading Jaynes on prior formation, and I'd love to see more posts here on the topic. Maybe I'll write one if I ever have the chance to get some reading done.

As far as this problem goes, I agree we have some information about other colors. I want to know what Omega counts as "red" though, because that will go a long way in determining what sort of prior we'd assign.

Based on my limited understanding of physics, if we assume the bead only reflects a single wavelength, then it would be red if the wavelength were between 620 and 750...

[-]loqi30

It's not like you have any information. Assuming you don't think Omega is out to deliberately screw with you

These are contradictory assumptions. I have no information. I have no idea what Omega is out to do. The whole point of invoking Omega is to obliterate meaningful priors. I'm with byrnema here, probabilities are a tool for making maximally effective use of information. Without any such information, the only correct answer to "what is your probability" is "I don't have one".

Interestingly, figuring out the answers to questions of this kind, basically about prior, we are dealing with issues similar to those in elicitation of human values. In both cases, the answer is hidden in our minds, never in explicit and consistent form, with no hope of constructing a precise model that will give the answer. The only way to approximate the solution is to consider arguments for and against, consider relared situations, think, and listen to your inner voice, to intuitive response that says that it's proper to save a child, and you agree, tha...

Alicorn, I think it'd be appropriate to add the following link at the beginning of the article:

Related to: Priors as Mathematical Objects.

Even if Omega had asked about the bead being lilac, and you'd dutifully given a tiny probability, it would not have surprised you to see a lilac bead emerge from the jar.

I see this conclusion as a mistake: being surprised is a way of translating between intuition and explicit probability estimates. If you are not surprised, you should assign high enough probability, and otherw...

7Simetrical
That's not true at all. Before I'm dealt a bridge hand, my probability assignment for getting the hand J♠, 8♣, 6♠, Q♡, 5♣, Q♢, Q♣, 5♡, 3♡, J♣, J♡, 2♡, 7♢ in that order would be one in 3,954,242,643,911,239,680,000. But I wouldn't be the least bit surprised to get it. In the terminology of statistical mechanics, I guess surprise isn't caused by low-probability microstates ― it's caused by low-probability macrostates. (I'd have been very surprised if that were a full suit in order, despite the fact that a priori that has the same probability.) What you define as a macrostate is to some extent arbitrary. In the case of bridge, you'd probably divide up hands into classes based on their utility in bridge, and be surprised only if you get an unlikely type of hand. In this case, I'd probably divide the outcomes up into macrostates like "red", "some other bright color like green or blue", "some other common color like brown", "a weird color like grayish-pink", and "something other than a solid-colored ball, or something I failed to even think of". Each macrostate would have a pretty high probability (including the last: who knows what Omega's up to?), so I wouldn't be surprised at any outcome. This is an off-the-cuff analysis, and maybe I'm missing something, but the idea that any low-probability event should be surprising certainly can't be correct.
Thank you, my mistake. I don't understand 'surprise'. Let's see... It looks like 'surprise' is something about promoting a new theory about the structure of environment that was previously dormant, forcing you to drop many cached assumptions. For example, if (surprise, surprise...) you win a lottery, you may promote a previously dormant theory that you are on a holodeck. If you are surprised by observing 1000 equal quantum coinflips (replicated under some conditions, with apparatus not to blame), you may need to reconsider the theory of physics. If you experience surprising luck in a game of dice, you start considering the possibility that dice are weighted.
0Cyan
... but it isn't, because the degree of surprise doesn't just depend on the raw probability, but also only the number of other possible outcomes under consideration. That Omega uses the term "lilac" may reasonably be taken as evidence that the space of color outcomes should be treated as finely divided. ETA: I guess the mistake is in comparing feelings of surprise across outcomes with the same probability embedded in event spaces with different cardinalities.
1JGWeissman
If Omega asked me the probability of the next bead being lilac, I would be surprised to if the next bead actually was lilac, in a way I would not be surprised to find the bead is turquoise, an event to which I assign equal probability, but was not specifically considering prior to the draw, as any higher probability set of events which excludes drawing a turquoise bead would seem artificial. If the first two beads are the colors Omega asks me about, my leading theory would be that Omega will draw out a bead of which ever color he just brought up. (The first draw would cause me to consider this with roughly equal probability as maximum entropy.)
0billswift
"doesn't just depend on the raw probability" - Correct. It also depends strongly on how reliable you think your estimate of the probability is. That is, your confidence interval.
Well, maybe it isn't, but it should.

But because you start with no information, it's very hard to gather more. Suppose Omega reaches into the jar and pulls out a red bead. Does your probability that the second bead will be red go up... down... [or] stay the same...?

My intuition here is to start with an uninformative prior over possible bead-generating mechanisms. (You still have the problem of how to divide up the state space, but that's nothing new.) If a red bead comes out first, I update the probabilities that I assign to each mechanism and proceed from there.

For me, the Omega problem described in the post presents the following conundrum: what is a probability in the limit of no information?

Suppose we employ a pragmatic perspective: the "probability of an event", as a mathematical object, is a tool that is used to summarize information about the past and/or future occurrence of that event. In the limit of no information, using the pragmatic view, there is no justification for assigning a probability not because we don't know what it is, but because it has no use in summarizing information.

If you do...

This post confused me enormously. I thought I must be missing something, but reading over the comments, this seems to be true for virtually all readers.

What exactly do you mean by "bead jar guess"? "Surprise"? "Actual probability"? Are you making a new point or explaining something existing? Are you purposely being obscure "to make us think"?

I propose replacing this entire post with the following text:

Hey everybody! Read E.T. Jaynes's Probability Theory: The Logic Of Science!

2Alicorn
By "bead jar guess" I mean a wild, nearly-groundless assignment of a probability to a proposition. This is as opposed to a solidly backed up estimate based on something like well-controlled sample data, or a guess made with an appeal to an inelegant but often-effective hack like the availability heuristic.
0talisman
Groundless or not, if you propoose to run two experiments X and Y, and select outcomes x of experiment X and y of experiment Y before running the experiments, and assign x and y the same probabilities, you have to be equally surprised by x occurring as you are by y occurring, or I'm missing something deep about what you're saying about probabilities. Are you using the word "probability" in a different sense than Jaynes?
1Alicorn
I haven't read Jaynes's work on the subject, so I couldn't say. However, if he thinks that equal probabilities mean equal obligation to be surprised, I disagree with him. It's easy to do things that are spectacularly unlikely - flip through a shuffled deck of cards to see a given sequence, for instance - that do not, and should not, surprise you at all.
5steven0461
"Surprise", as I understand it, is something rational agents experience when an observation disconfirms the hypothesis they currently believe in relative to the hypothesis that "something is going on", or the set of unknown unknowns. If you generate ten numbers 1-10 from a process you think is random, and it comes up 5285590861, that is no reason to be surprised, because the sequence is algorithmically complex, and the hypothesis that "something is going on" assigns it a conditional probability no higher than the hypothesis that the process is random. But if it comes up 1212121212, that is reason to be surprised, because the sequence is algorithmically simple, so the hypothesis that "something is going on" assigns it higher conditional probability than the hypothesis that the process is random. The surprised agent is then justified in sitting up and expending resources trying to gather more info.
4andrewc
1. Point your browser at amazon 2. Order ETJ's book. 3. Wait approx one week for delivery 4. Read it. I don't mean to sound gushing but Jayne's writing on probability theory is the clearest, most grounded, and most entertaining material you will ever read on the subject. Even better than that weird AI dude. Seriously it's like trying to discuss the apocalypse without reading Revelations...
0[anonymous]
I tend to agree. If I discovered that Jaynes had said such a thing I would be very surprised indeed. I'll be surprised when the probability of seeing something with a that probability or less occur is low.
0talisman
That's because you didn't specify the sequence ahead of time, right?
0Alicorn
Writing down a sequence ahead of time makes it more interesting when it turns up, not more unlikely. Given the possibility of cheating, it might make it more likely.
0talisman
Belated apologies for cranky tone on this comment.

Thinking about how an Occamian learner like AIXI would approach the problem, it would probably start from the simplest domain theory "beads have a color, red is a color I've heard mentioned, therefore all beads are red", p=1. If the first bead was grey, it would switch to "all beads are grey", p=0. The second bead is red, "half and half", p = 0.5, and so on, ratcheting up theories from the simplest first.

2Peter_de_Blanc
FYI, AIXI does not work like this; it uses a probability distribution over all Turing machines.
This is not how AIXI [*] works. It considers all possible programs at the start, with some probability. The simplest program that fits the data is not the only one it considers; it just gets most of the probability mass. So, from the start, it will give some tiny probability to a hypothesis that the beads will spell War and Peace is morse code. Only when this hypothesis is falsified by the data, it will drop out of race. [*] M. Hutter (2003). `A Gentle Introduction to The Universal Algorithmic Agent AIXI'. Tech. rep. [abstract/download]
1MBlume
and thus his bayes-score drops to -Infinity
0JulianMorrison
I don't think AIXI tries to maximize its Bayes score in one round - it tries to minimize the number of rounds until it converges on a good-enough model.

I think this post could have been more formally worded. It draws a distinction between two types of probability assignment, but the only practical difference given is that you'd be surprised if you're wrong in one case but not the other. My initial thought was just that surprise is an irrational thing that should be disregarded ― there's no term for "how surprised I was" in Bayes' Theorem.

But let's rephrase the problem a bit. You've made your probability assignments based on Omega's question: say 1/12 for each color. Now consider another situ...

6saturn
Not quite. The intuitive notion of "how surprised you were" maps closely to bayesian likelihood ratios. Regarding your die/beads scenarios: In your die scenario, you have one highly favored model that assigns equal probability to each possible number. In the beads scenario you have many possible models, all with low probability; averaging their predictions gives equal probability to each possible color. To simplify things, let's say our only models are M, which predicts the outcomes are random and equally likely (i.e. a fair die or jar filled with an even ratio of 12 colors of beads), and not-M (i.e. a weighted die or jar filled with all the same color beads). In the beads scenario we might guess that P(M)=.1; in the die scenario P(M)=.99. In both cases, our probability of red/one is 1/12, because neither of our models tell us which color/number to expect. But our probability of winning the bet is different -- we only win if M is correct.
0Simetrical
That clears things up a lot. I hadn't really thought about the multiple-models take on it (despite having read the "prior probabilities as mathematical objects" post). Thanks.
2GuySrinivasan
No. As another (yours is one) simple counterexample, if I flip a fair coin 100 times you expect around 50 heads, but if I either choose a double-head or double-tail coin and flip that 100 times, you expect either 100 heads or 100 tails - and yet the probability of the first flip is still 50/50. A distribution over models solves this problem. IIRC you don't have to regress further, but I don't remember where (or even if) I saw that result.
1orthonormal
To clarify: if you know Guy chose either a double-head or double-tail coin, but you have no idea which, then you should assign 50% to heads on the first flip, then either 0% or 100% to heads after, since you'll the know which one it was. It's been linked too often already in this thread, but the example in Priors as Mathematical Objects neatly demonstrates how a prior is more than just a probability distribution, and how Simetrical's question doesn't lead to paradox.
0[anonymous]
The justification given in the original post was spectacularly wrong. The assignments themselves may not be. One could just as easily be using the shorthand for "slightly more than 1/12 because I now know that red is a color Omega considers 'color-worthy', he can see that I've got red receptive cones in my eyes and this influences my probability a little more than the possibility that he has obscure color beads. And screw it. Lilac is freaking purple anyway. And he asked for my probability, not that of some pedantic ponce!"

I'd say like the second example, most of our beliefs are bead jar guesses informed by untrustworthy informants. Namely our parents and other adults around when we were young.

0MrHen
FYI, italics can be entered as such: *this will be italic*. There are more formatting tips available by clicking the "help" link under the comments box.

Do you see this as being sort of like Jimmy's metauncertainty?

Also, if Omega pulled out a bead and then asked you about the next one, the Rule of Succession would be a good place to start making guesses.

0Alicorn
It's a little like metauncertainty, except that my post has much less frightening math.

Omega asking you that question, "What's the probability that the bead will be red" is itself information about the beads - Omega is more likely to ask that question in cases where the color is relevant.

To do things properly, you could suppose that Omega is inspiring himself from existing bead-color-guessing logic puzzles (there are plenty of examples of those in human history), and you could assign probabilities to each type of guessing games, noting how many types of beads there are etc. You can also collect statistics about which combinations o...

0Alicorn
I'm not sure what you mean by this. Obviously the color is relevant, in that it's what he wants you to guess at - but the fact that he asked about red and not about brown is not suggestive in any way. This is Omega we're talking about, not someone with normal psychology.

Apparently, the term you are searching for is "Second Order Probability".

See here for a paper: www.dodccrp.org/events/2000_CCRTS/html/pdf_papers/Track_4/124.pdf

D. Bamber and I.R. Goodman, (2000). New uses of second order probability techniques in estimating critical probabilities in command & control decision-making. [abstract] [pdf]. Please cite at least the name of the paper, preferably add a link to its abstract. It's not apparent that second order probability is directly related to this article.
0Alicorn
The link gives me a 404.
[-][anonymous]00

I've just started reading Jaynes on prior formation, and I'd love to see more posts here on the topic. Maybe I'll write one if I ever have the chance to get some reading done.

As far as this problem goes, I agree we have some information about other colors. I want to know what Omega counts as "red" though, because that will go a long way in determining what sort of prior we'd assign.

Based on my limited understanding of physics, if we assume the bead only reflects a single wavelength, then it would be red if the wavelength were between 620 and 750...

I'm sure not sure I understand the point of this post. Are you saying that guesses without any information are inherently unfounded?

How would guessing 50% on the first pull be any worse since, by definition of the problem, you have no information? As soon as you have seen one bead, however, you have perfect historical information which is better than none.

Assuming that Omega is picking at random, it makes sense to me to simply pick a random percentage on the first pull and then swing to 0% or 100% once you see a bead. Update again on the second bead to ...

2Alicorn
You should not guess that the first bead has a 50% chance of being red, because if you do, you can have this conversation: Omega: What is the probability of the first bead being red as opposed to non-red? You: Fifty-fifty. Omega: So you would consider it more than fair if I offered you three dollars if the bead is red, and you paid me a dollar if it was non-red? You: Sure, I'll take that bet. Omega: What is the probability of the first bead being blue as opposed to non-blue? You: Fifty-fifty. Omega: So you would consider it more than fair if I offered you three dollars if the bead is blue, and you paid me a dollar if it was non-blue? You: Sure, I'll take that bet. (...and so on for ten more colors.) Omega pulls out a red bead. He owes you three dollars, but you owe him eleven dollars. He wins.
2Zvi
You could have that conversation, but you don't have to. The argument for assigning 50% to red is that it's the only question Omega has asked you. There are several ways out of that. The first one is that the moment he offers you a 25% bet I would update to presume that 3:1 is not a positive e.v. bet, with a new number of perhaps 12.5% with a range of 0% to 25% with symmetric distribution. Similarly, if he offered me three to one that it wasn't red, I would presume that it probably will be. On a similar note, when he asks about blue (even without any bets involved) I can't see answering higher than 33.3%. Contrast this with Alicorn watching this incident and offering me 3:1 after Omega asks my probability for red and I say 50%. I still have to update for Alicorn's opinion, but I might or might not accept that bet.
The estimate should take into account the expectation of being asked further questions. The ignorance prior is applied to a model of observation. The model of observation expresses which questions you may be asked, and what structure will the dependencies between these possible observations have.
0MrHen
1JamesAndrix
1jimrandomh
Not every question Omega could ask would provide new information, but some certainly do. Suppose his follow-up questions were "What is the probability that the bead is transparent?", "What is the probability that the bead is made of wood?" and "What is the probability that the bead is striped?". It is very likely that your original probability distribution over colors implicitly set at least one of these answers to zero, but the fact that Omega has mentioned it as a possibility makes it considerably more likely.
0JamesAndrix
0conchis
There's no need to, because probability is in the mind.
0JamesAndrix
If you're going to update based on what omega asks you then you must believe there is a connection that you have some information about. If we don't know anything about omega's thought process or goals, then his questions tell us nothing.
0conchis
I think our only disagreement is semantic. If I initially divide the state space into solid colours, and then Omega asks if the bead could be striped, then I would say that's a form of new information - specifically, information that my initial assumption about the nature of the state space was wrong. (It's not information I can update on; I have to retrospectively change my priors.) Apologies for the pointless diversion.
An ideal model of the real world must allow any miracle to happen, nothing should be logically prohibited.
0MrHen
Of note, I was operating under a bad assumption with regards to the original example. I assumed that the set was a finite but unknown set of colors or an infinite set of colors. In the former case, every question is giving a little information about the possible set. In the latter it really does not matter much. Yes, this is true. Personally, I am still curious about what to do with the two hundred color questions.
Don't think of probability as being mutable, as getting updated. Instead, consider a fixed comprehensive state space, that has a place on it for every possible future behavior, including the possible questions asked, possible pieces of evidence presented, possible actions you make. Assign a fixed probability measure to this state space. Now, when you do observe something, this is information, an event, a subset on the global state space. This event selects an area on it, and encompasses some of the probability mass. The statements, or beliefs (such as "the ball #2 will be red"), that you update on this info, are probabilistic variables. A probabilistic variable is a function that maps the state space on a simpler domain, for example a binary discrete probabilistic variable is basically an event, a subset of the state space (that is, in some states, the ball #2 is indeed defined to be red, these states belong to the event of ball #2 being red). Your info about the world retains only the part of the state space, and within that part of the state space, some portion of the probability mass goes to the event defining your statement, and some portion remains outside of it. The "updating" only happens when you focus on this info, as opposed to the whole state space. If that picture is clear, you can try to step back to consider what kind of probability measure you'd assign to your state space, when its structure already encodes all possible future observations. If you are indifferent to a model, the assignment is going to be some kind of division into equal parts, according to the structure of state space.
0orthonormal
IAWYC, but as pedagogy it's about on the level of "How should you imagine a 7-dimensional torus? Just imagine an n-dimensional torus and let n go to 7." Eliezer's post on priors explains the same idea more accessibly. EDIT: Sorry, I didn't notice you already linked it below.
1Alicorn
What if Omega wants you to commit to a bet based on your probabilities at every step? Or what if he just straight up asks you what color you want to guess the bead will be, without asking about any individual colors? (Then you'd probably be best served by switching to a language with fewer basic color words, but that aside...)
0MrHen
4Eliezer Yudkowsky
MrHen, whatever strategy you're employing here, it doesn't sound like a strategy for arriving at the really truly correct answer, but some sort of clever set of verbal responses with a different purpose entirely. In real life, just because Omega asked if the bead is red simply does not mean there is probability 0 of it being green.
3MrHen
0soreff
Is Omega privileging the hypothesis that the bead is red?:-)
2JGWeissman
Me: No, because you have more information than I do, and the fact that you would offer this bet is evidence that I should use to update my epistemic probabilities.
0Alicorn
Well, Omega doesn't really need the money. There's no reason to believe he would balk at offering you a more-than-fair bet.
0JGWeissman
Well, in the case of Omega, I would at least suspect that he intends to demonstrate that I am vulnerable to a Dutch book, even though he doesn't need the money.
8Nominull
If you meet an Omega, that is pretty good evidence that you are living in a simulation: specifically, you are being simulated inside a philosopher's brain as a thought experiment.
1cousin_it
Sorry, you haven't convincingly demonstrated the wrongness of 50%. MrHen's position seems to me quite natural and defensible, provided he picks a consistent prior. For example, I'd talk with Omega exactly as you described up to this point: ...Omega: What is the probability of the first bead being blue as opposed to non-blue? Me: 25%. You ask why 25%? My left foot said so... or maybe because Omega mentioned red first and blue second. C'mon Dutch book me.
-1Alicorn
I think that doing it this way assumes that Omega is deliberately screwing with you and will ask about colors in a way that is somehow germane to the likelihood. Assume he picked "red" to ask about first at random out of whatever colors the beads come in.
1cousin_it
This new information gives me grounds to revise my estimates as Omega asks further questions, but I still don't see how it demonstrates the wrongness of initially answering 50%.
2MrHen
The reason 50/50 is bad is because the beads in the jar come in no more than 12 colors and we have no reason to favor red over the other 11 colors. Knowing there is a cap of 12 possible options, it makes intuitive sense to start by giving each color equal weights until more information appears. (Namely, whenever Omega starts pulling beads.)
2cousin_it
We have a reason: Omega mentioned red.
0MrHen
I suppose the relevant question is now, "Does Omega mentioning red tell us anything about what is in the jar?" When we know the set of possible objects in the jar, it really tells us nothing new. If the set of possible objects is unknown, now we know red is a possibility and we can adjust accordingly. The assumption here is that Omega is just randomly asking about something from the possible set of objects. Essentially, since Omega is admitting that red could be in the jar, we know red could be in the jar. In the 12 color scenario, we already know this. I do not think that Omega mentioning red should effect our guess.
3cousin_it
All this arguing about priors eerily resembles scholastics, balancing angels on the head of a pin. Okay I get it, we read Omega's Bible differently: unlike me, you see no symbolic significance in the mention of red. Riiiiight. Now how about an experiment?
0MrHen
Agreed. For what it is worth, I do see some significance in the mention of red, but cannot figure out why and do not see the significance in the 12 color example. This keeps setting off a red flag in my head because it seems inconsistent. Any help in figuring out why would be nifty. In terms of an experiment, I would not bet at all if given the option. If I had to choose, I would choose whichever option costs less and right it off as a forced expense. In English: If Omega said he had a dollar claiming the next bead would be red and asked me what I bet I would bet nothing. If I had to pick a non-zero number I would pick the smallest available. But that doesn't seem very interesting at all.
0Peter_de_Blanc
Then each time Omega mentions another color, it increases the expected number of colors the beads come in.
0MrHen
I think Alicorn is operating under a strict "12 colors of beads" idea based on what a color is or is not. As best as I can tell, the problem is essentially, "Given a finite set of bead colors in a jar, what is the probability of getting any particular color from a hidden mixture of beads?" The trickiness is that each color could have a different amount in the jar, not that there are any number of colors. Alicorn answered elsewhere that when the jar has an infinite set of possible options the probability of any particular option would be infinitesimal.
0orthonormal
If the number of possible outcomes is finite, fixed and known, but no other information is given, then there's a unique correct prior: the maxentropy prior that gives equal weight to each possibility. (Again, though, this is your prior before Omega says anything; you then have to update it as soon as ve speaks, given your prior on ver motivations in bringing up a particular color first. That part is trickier.)
0MrHen
How would you update given the following scenarios (this is assuming finite, fixed, known possible outcomes)? 1. Omega asks you for the probability of a red bead being chosen from the jar 2. Omega asks you for the probability of "any particular object" being chosen 3. Omega asks you to name an object from the set and then asks you for the probability of that object being chosen
1orthonormal
I don't think #2 or #3 give me any new relevant information, so I wouldn't update. (Omega could be "messing with me" by incorporating my sense of salience of certain colors into the game, but this suspicion would be information for my prior, and I don't think I learn anything new by being asked #3.) I would incrementally increase my probability of red in case #1, and decrease the others evenly, but I can't satisfy myself with the justification for this at the moment. The space of all minds is vast; and while it would make sense for several instrumental reasons to question first about a more common color, we're assuming that Omega doesn't need or want anything from this encounter. In the real-life cases which this is meant to model, though, like having a psychologist doing a study in place of Omega, I can model their mind by mine and realize that there are more studies in which I'd ask about a color I know is likely to come up, than studies in which I'd pick a specific less-likely color, and so I should update p(red) positively. But probably not all the way to 1/2.
0Cyan
To make consistent bets, we need a prior on the number of possible outcomes.
0Alicorn
The fact that Omega is speaking English and uses the word "red" as opposed to "scarlet" or something is decent evidence that there are twelve colors in beadspace.
0Cyan
What happens if you've taken bets on twelve colors [ETA: that eat up all your probability] and then Omega asks you to name odds on a transparent bead?
0Alicorn
I was operating under the assumption that clear is not a "solid color".
0Cyan
Right you are. I didn't read the original problem carefully enough... Nevertheless, you can replace "transparent" with a surprising color like lilac, fuchsia, or, um, cyan to restore the effect. The point is that even decent evidence that there are twelve colors in beadspace doesn't justify a probability distribution on the number of colors that places all of its mass at twelve.
1Alicorn
The twelve basic colors are so called because they are not kinds of other colors. Lilac and fuchsia are kinds of purple (I guess you could argue that fuchsia is a kind of red, instead, but pretend you couldn't), and cyan is a kind of blue. Even if you pull out a navy bead and then a cyan bead, they are both kinds of blue in English; in Russian, they would be different colors as unalike as pink and red.
0Cyan
So you're arguing that by definition, the basic color words define a mutually exclusive and exhaustive set. But there are colors near cyan which are not easy to categorize -- the fairest description would be blue-green. In the least convenient world, when Omega asks you for odds on blue-green, you ask it if that color counts as blue and/or green, and it replies, "Neither; I treat blue-green as distinct from blue and green." Then what do you do?
0Alicorn
I was mentally categorizing that as "Omega deliberately screwing with you" by using English strangely, but perhaps that was unmotivated of me. But this gets into a grand metaphysical discussion about where colors begin and end, and whether there is real vagueness around their borders, and a whole messy philosophy of language hissy fit about universals and tropes and subjectivity and other things that make you sound awfully silly if you argue about them in public. I ignored it because the idea of the post wasn't about colors, it was about probabilities.
1Cyan
That's a shame, because uncertainty about the number of possible outcomes is a real and challenging statistical problem. See for example Inference for the binomial N parameter: A hierarchical Bayes approach (abstract)(full paper pdf) by Adrian Raftery. Raftery's prior for the number of outcomes is 1/N, but you can't use that for coherent betting.
0JamesAndrix
I think there's also the question of inferring the included name space and possibility space from the questions asked. If he asks you about html color #FF0000 (which is red) after asking you about red, do you change your probability? Assuming he's using 12 color words because he used 'red' is arbitrary. Even with defined and distinct color terms, the question is, what of those colors are actual possibilities (colors in the jar) as opposed to logical possibilities (colors omega can name) and I think THAT ties back to Elizer's article about Job vs. Frodo.
0MrHen
Personally, I think the intent has less to do with classifying colors strangely and more to do with finding a broader example where even less information is known. The misstep I think I took earlier had to do with assuming that the colors were just part of an example and the jar could theoretically hold items from an infinite set. I get that when picking beads from the set of 12 colors it makes sense to guess that red will appear with a probability near 1/12. An infinite set, instead of 12, is interesting in terms of no information as well. As far as I can tell, there is no good argument for any particular member of the set. So, asking the question directly, what if the beads have integers printed on them? What am I supposed to do when Omega asks me about a particular number?
0Alicorn
Unless you have a reason to believe that there is some constraint on what numbers could be used - if only a limited number of digits will fit on the bead, for example - your probability for each integer has to be infinitesimal.
1orthonormal
You're not allowed to do that. With a countably infinite set, your only option for priors that assign everything a number between 0 and 1 is to take a summable infinite series. (Exponential distributions, like that proposed by Peter above, are the most elegant for certain questions, but you can do p(n)=cn^{-2} or something else if you prefer to have slower decay of probabilities.) In the case with colors rather than integers, a good prior on "first bead color, named in a form acceptable to Omega" would correspond to this: take this sort of distribution, starting with the most salient color names and working out from there, but being sure not to exceed 1 in total. Of course, this is before Omega asks you anything. You then have to have some prior on Omega's motivations, with respect to which you can update your initial prior when ve asks "Is it red?" And yes, you'll be very metauncertain about both these priors... but you've got to pick something.
0MrHen
I am happy with that explanation. Thanks.
1Peter_de_Blanc
Why not, say, p(n) = (1/3) * 2^(-|n|)?
0MrHen
If p(n) = (1/3) * 2^(-|n|), then: * p(1) = (1/3) * 2^(-1) = 0.166666667 * p(86) = (1 / 3) * (2^(-86)) = 4.30823236 × 10^(-27) * p(1 000 000) = (1/3) * 2^(-1 000 000) = Lower than Google's calculator lets me go Are you willing to bet that 1 is going to happen that much more often than 1,000,000?
2GuySrinivasan
The point is that your probability for the "first" integers will not be infinitesimal. If you think that drops off too quickly, instead of 2 use 1+e or something. p(n) = e/(e+2) * (1+e)^(-|n|). And replace n with s(n) if you don't like that ordering of integers. But regardless, there's some N for which there is an n with |n|N such that p(n)/p(m) >> 1.
0Peter_de_Blanc
I wasn't talking about limiting frequencies, so don't ask me "how often?" Would you bet \$1 billion against my \$1 that no number with absolute value smaller than 3^^^3 will come up? If not then you shouldn't be assigning infinitesimal probability to those numbers.
0MrHen
To dispel this confusion, you should read on algorithmic information theory.
0MrHen
Is there a good place to start online? Can I just Google "algorithmic information theory"?
1Cyan