William_Kasper

My Bayesian Enlightenment

Mr. Bonaccorsi:

Here are two links to classic posts by Eliezer Yudkowsky that you may find pertinent to the second dialog from your last comment. I hope you enjoy them.

My Bayesian Enlightenment

Let's establish some notation first:

P(H): My prior probability that the coin came up heads. Because we're assuming that the coin is fair before you present any evidence, I assume a 50% chance that the coin came up heads.

P(H|E): My posterior probability that the coin came up heads, or the probability that the coin came up heads, given the evidence that you have provided.

P(E|H): The probability of observing what we have, given the coin in question coming up heads.

P(E&H): The probability of you observing the evidence and the coin in question coming up heads.

P(E&-H): The probability of you observing the evidence and the coin in question coming up tails.

P(E): The unconditional probability of you observing the evidence that you presented. Because the events (E&H) and (E&-H) are mutually exclusive (one cannot happen at the same time as the other) and the events (H) and (-H) are collectively exhaustive (the probability that at least one of these events occurs is 100%), we can calculate P(E):

P(E) = P(E&H) + P(E&-H)

P(E) = P(E|H) P(H) + P(E|-H) P(-H)

Using Bayes' Theorem, we can calculate P(H|E) after we determine P(E|H) and P(E|-H):

P(H|E) = [P(E|H) P(H)] / [P(E|H) P(H) + P(E|-H) P(-H)]

Let me try this. You come upon a man who, as you watch, flips a 50-50 coin. He catches and covers it; that is, the result of the flip is not known. I, who have been standing there, present you the following question: "What is the chance the coin is heads?"

In this case we can assume that our lack of knowledge is independent of the result of the coin toss; P(E|H) = P(E) = P(E|-H). So

P(H|E) = P(E) (50%) / [P(E) (50%) + P(E) (1 - 50%)] = [P(E) / P(E)] (50% /100%) = 50%.

The next day, you come upon a different man, who, as you watch, flips a 50-50 coin. Again, he catches it; again, the result is not revealed. I, who have been standing there, address you as follows: "Just before you arrived, that man flipped that same coin; it came up heads. What is the chance it is now heads?"

Again here, your probability of observing the first result is independent of the second result. So P(H|E) = 50%.

You come upon a man who is holding a 50-50 coin. I am with him. There is the following exchange:

I (to you, re the man with the coin): This man has just flipped this coin two times.

You: What were the results?

I: One of the results was heads. I don’t remember what the other was.

Here we can note that there are four mutually exclusive, collectively exhaustive, and equiprobable outcomes. Let's call them (HH), (HT), (TH), and (TT), where the first of the two symbols represents the result that you remember observing. Given that you remember observing a result of heads, our evidence is (HH or HT). The second coin is heads in the case of (HH), which is as probable as (HT). Given that P(HH) = P(HT) = 25%, P(HH or HT) = 50%

P(HH|HH or HT) = P(HH or HT|HH) P(HH) / P(HH or HT)

P(HH|HH or HT) = 1 (25% / 50%) = 50%

After I tell you that one of the results was heads but that I don't remember what the other was, you say: "Which do you remember, the first or the second?" I reply, "I don’t remember that either."

We can use the same method as in Question C. Since the ordinality of the missed observation is independent from the result of the missed observation, the probability is the same as in Question C, which is 50%.

My Bayesian Enlightenment

No, the chance that the kidnapped child is a boy is 1/2.

In the correct version of this story, the mathematician says "I have two children", and you ask, "Is at least one a boy?", and she answers "Yes". Then the probability is 1/3 that they are both boys.

In the correct version of the story, you do not gain access to any information that allows you to differentiate between the mathematician's two children and identify a specific child as a boy.

A woman says, "I have two children." You respond, "What are their sexes?" She says, "At least one of them is a boy. The other was kidnapped before I was informed of its sex."

In your story, you are able to partition the woman's children into "the kidnapped one" and "the other one", and the woman provides you with the information that "the other one" is a boy. The sex of "the kidnapped one" is independent of the sex of "the other one". That is,

P("the kidnapped one" is a boy | "the other one" is a boy") = P("the kidnapped one" is a boy)

The Worst Problem You've Ever Encountered and Solved. And the One You Didn't, Yet!

But why is the goal of voting an example important? For me, what matters is creating your own example, and helping those who put theirs.

I agree with you. Receiving votes on our posts and comments is only an instrument to help us build better content. The content and how people use it is what matters.

Although the karma voting system provides imperfect information, it provides cheap imperfect information. Separating the question and answer seems like an easy way to make better use of the information that the votes provide. One benefit that I see from the separation is that you receive slightly more detailed feedback, like in a case where some people might upvote your post because of the thoughts that your question provokes but others may downvote the post because they take issue with your example. If enough people downvote the post because of the answer despite the quality of discussion that they think your question provides (which seems pretty unlikely), the post might become buried because of its low rating.

The Worst Problem You've Ever Encountered and Solved. And the One You Didn't, Yet!

Or you could put your answer in the body as an example, explaining that you've also posted the answer as a comment. Then people can vote on your answer independently from your question, and you can establish the expected form before people begin reading other people's comments.

Trust in Bayes

rstarkov wrote a nice discussion piece on the two envelopes problem: Solving the two envelopes problem. thomblake commented that the error most people make with this problem is treating the amounts of money in the envelopes as fixed values when calculating the expectation.

Solving the two envelopes problem

Here's a solution to a more general version of the problem:

Let's say that the red envelope contains *N* times as much money as the blue envelope with probability *p*, and the blue envelope contains *N* times as much money as the red envelope with probability (1 - *p*).

Without loss of generality, *N* is at least 1.

If *N* = 1, both envelopes contain the same amount, and there is no point in switching.

If *N* > 1, let the variable *s* represent the smaller amount of money between the amounts of the two envelopes. So one envelope contains *s*, and the other envelope contains *Ns*.

Scenario 1: The blue envelope contains *s*, and the red envelope contains *Ns*. This scenario occurs with probability *p*.

Scenario 2: The red envelope contains *s*, and the blue envelope contains *Ns*. This scenario occurs with probability (1 - *p*).

Assume *s* is not less than 0.

The expected amount of money in the blue envelope, E(B) = *sp* + *Ns*(1 - *p*) = *s*(*p* + *N* - *Np*).

The expected amount of money in the red envelope, E(R) = *s*(1 - *p*) + *Nsp* = *s*(1 - *p* + *Np*).

If *s* = 0, both envelopes contain the same amount, and there is no point in switching.

If *s* > 0, compare the expectations of the amounts of money in the envelopes in terms of *s*:

E(B) > E(R) when

*s*(*p* + *N* - *Np*) > *s*(1 - *p* + *Np*)

*p* + *N* - *Np* > 1 - *p* + *Np*

2*p* - 2*Np* > 1 - *N*

2*p*(1 - *N*) > 1 - *N*

2*p* < 1, because (1 - *N*) < 0

*p* < 1/2

Similarly, E(B) < E(R) when *p* > 1/2, and E(B) = E(R) when *p* = 1/2.

The ratios of the expected amounts of money in each envelope depend only on *p* when *s* > 0 and *N* > 1.

Both envelopes contain the same amount of money when *s* = 0, *N* = 1, or both.

So if your degree of belief that the red envelope contains more money than the blue envelope is the same as your degree of belief that the blue envelope contains more money than the red envelope, don't bother switching, unless you need to kill time (which you already knew intuitively, but *Q.E.D.*)

I got the idea of using the variable *s* to represent the smaller of the amounts in the two envelopes from R Falk 2008: "The Unrelenting Exchange Paradox".

Rationality Quotes May 2012

[Political "gaffe" stories] are completely information-free news events, and they absolutely dominate political news coverage and analysis. It's like asking your doctor if the X-rays show a tumor, and all he'll talk about is how stupid the radiologist's haircut looks. . . . ["Blast"] stories are. . . just as content-free as the "gaffe" stories. But they are popular for the same reason: There's a petty, tribal satisfaction in seeing a member of our team really put the other team in their place. And there's a rush of outrage adrenaline when the other team says something mean about us. So, instead of covering pending legislation or the impact it could have on your life, the news media covers the dick-measuring contest.

-David Wong, 5 Ways to Spot a B.S. Political Story in Under 10 Seconds

Rationality Quotes May 2012

It's weird how proud people are of not learning math when the same arguments apply to learning to play music, cook, or speak a foreign language.

"Your dirty lying teachers use only the midnight to midnight 1 day (ignoring 3 other days) Time to not foul (already wrong) bible time."