Alice and Bob are sitting on the opposite sides of a table. Bob closes his eyes while Alice throws two coins on the table. Alice covers them with her hands and Bob opens his eyes.
Now Alice can ask Bob "What is the probability that they are both Heads?". Without knowing anything about the coins, it seems logical for Bob to assume they are fair and answer "1/4".
Helping (confusing) Bob
Consider several scenarios where Alice gives additional information to Bob. How would that affect Bob's response in each scenario?
Alice reveals to Bob "The coin on your left side is Heads.".
Bob can make the following argument: With the new information, the following possibilities remain: "H H" and "H T". Bob concludes that the probability is now "1/2".
Alice reveals to Bob "The coin under my left hand is Heads.".
Bob can make an analogous argument to the previous one (leaving "H H" and "T H" as possibilities) to conclude the probability is now "1/2".
Alice reveals to Bob "The left coin is Heads.".
There are two arguments that Bob could make:
Bob doesn't know whether Alice meant his or her left. In case she meant his left, the probability is now "1/2". In case she meant her left, the probability is now also "1/2". Thus, Bob concludes that the probability is now "1/2".
One of the coins is Heads but Bob doesn't know which one, so there are three possibilities: "H H", "H T", "T H". Thus, Bob concludes that the probability is now "1/3".
We get two different results - something is wrong!
Understanding the Confusion
If we don't know what Alice means by "left", the information we get from her is "one of the coins is H". The statement "one of the coins is H" is true if and only if the statement "the configuration is HH or HT or TH" is true. However, note that if Bob hears "the configuration is HH or HT or TH", he would never think to use the first argument. Let's examine the reasoning in more detail:
In case she meant his left, the probability becomes "1/2". In case she meant her left, the probability also becomes "1/2".
Assuming that she meant his left with probability 1/2 (thus her left also with probability 1/2), we get:
* She meant his left; the other coin is H (thus H H) - probability 1/4
* She meant his left; the other coin is T (thus H T) - probability 1/4
* She meant her left; the other coin is H (thus H H) - probability 1/4
* She meant her left; the other coin is T (thus T H) - probability 1/4
Now, let's examine the second argument:
One of the coins is Heads but Bob doesn't know which one, so there are three possibilities: "H H", "H T", "T H"
When Bob counts the number of outcomes, he is making the assumption that each outcome is equally likely. This seems justified - after all, for fair coins each of the four outcomes is equally likely to begin with.
Notice that we are counting the H H configuration twice in the first case and only once in the second case. This is what leads to the discrepancy, but both arguments seem reasonable. We should be allowed to examine the cases of "his left" and "her left" separately and aggregate the results. We should also be allowed to consider the possible configurations, given that there is H under one of Alice's hands. How come these two arguments lead to different probabilities?
The issue is quite subtle. Hypothetically, consider: what would Alice have said if the coins had landed T T? Maybe she would have said "the left coin is T" or maybe she would have said "both coins landed the same". The point is, Bob doesn't really know how Alice decides what to say - this is the key part of the puzzle. Imagine Alice says "the left coin is H" only for H H and says "the left coin is T" otherwise. If Bob knows this about Alice, then hearing "the left coin is H" would indicate P(H H) = 1.
If Bob knows that Alice picks a hand at random and says what she hides under that hand, this leads to probability 1/2.
If Bob knows that Alice indicates a hand in which she has H whenever she has at least one H, this leads to probability 1/3.
Another way to look at it is that the factual statement "one of the coins is H" does not carry the same information as "Alice said one of the coins is H". The probability Bob assigns to H H depends on what Bob knows about Alice. But maybe Bob just met Alice and knows nothing about her. If you were in Bob's place, how would you come up with a reasonable answer?
Asking Bayes for Help
Let us reason more formally. We will apply Bayesian reasoning to the problem. If we assume all coin toss outcomes are equally probable and then update on evidence, we should get a reasonable answer. Initially, there are 4 hypotheses: HH, HT, TH, TT.
P(HH) = 0.25
P(HT) = 0.25
P(TH) = 0.25
P(TT) = 0.25
We observe the evidence
observation := Alice says "The left coin is Heads."
We need to multiply the probability of each hypothesis by the likelihood of the observation (and normalize).
P(HH) P(observation | HH)
P(HT) P(observation | HT)
P(TH) P(observation | TH)
P(TT) P(observation | TT)
Now is the key question - what is the probability of Alice saying "The left coin is Heads." under the different hypotheses?
These probabilities could be pretty much anything - imagine that Alice is a liar and utters the phrase "The left coin is Heads." when both coins are T and remains silent otherwise. In such case, we would have P(observation | TT) = 1 and all other likelihoods being 0. One more reasonable assumption would be that Alice says what is the coin on Bob's left side. In that case, `P(observation | HH) = P(observation | HT) = 0.5` and `P(observation | TH) = P(observation | TT) = 0`. After multiplying the probabilities by the likelihood and normalizing, we get the posterior probability:
P(HH | observation) = 0.5
P(HT | observation) = 0.5
P(TH | observation) = 0
P(TT | observation) = 0
Note that this is just one possibility for how Alice decides what to say.
Priors for the Observation
We can model Alice as running an algorithm that takes the coin configuration as input and outputs the phrase which Alice utters. The algorithm could be something like this:
def say(configuration): if configuration == "HT": return "The left coin is Heads." elif configuration == "TH": return "The right coin is Heads." else: # configuration == "HH" or configuration == "TT" return "Both coins are the same."
Alice can run any of an infinite number of algorithms :
If we assign some prior probability to each algorithm, we can calculate the likelihood (note that the algorithm Alice runs and the coin toss are independent):
The logic is the same for the other hypotheses as well. The first term in the product
can be determined by examining the algorithm . Basically, we check whether (the probability is thus either 1 or 0). It seems that we have just moved the difficulty to determining - this is true in a sense, but at least we have some idea how to reason about such probabilities.
Consider Occam's razor:
The simplest explanation is usually the best one.
In our context we can say that shorter programs are more likely than longer ones. This doesn't give us an answer, it only explains how to approach the problem. In practice, we would need to make some assumptions to get probabilities for the different algorithms ("Alice tells the truth", "Alice always gives relevant information", etc.). In a sense, the problem is underdefined because no prior is specified and we need to assume one.
Many people argue over the Boy or girl paradox, which is basically another variant of what we examine here.
If all this seems a little complicated, it's because Bayesian reasoning is a general framework which can be applied to all reasoning under uncertainty. Humans are complicated and here we need to reason how a human acts. My goal is not to give you the answer (because the problem is underdefined) but to explain where the paradox comes from and the difficulty of the task.
Alice and Bob are sitting on the opposite sides of a table. Bob closes his eyes while Alice throws two coins on the table. Alice covers them with her hands and Bob opens his eyes.
Now Alice can ask Bob "What is the probability that they are both Heads?". Without knowing anything about the coins, it seems logical for Bob to assume they are fair and answer "1/4".
Helping (confusing) Bob
Consider several scenarios where Alice gives additional information to Bob. How would that affect Bob's response in each scenario?
Alice reveals to Bob "The coin on your left side is Heads.".
Bob can make the following argument: With the new information, the following possibilities remain: "H H" and "H T". Bob concludes that the probability is now "1/2".
Alice reveals to Bob "The coin under my left hand is Heads.".
Bob can make an analogous argument to the previous one (leaving "H H" and "T H" as possibilities) to conclude the probability is now "1/2".
Alice reveals to Bob "The left coin is Heads.".
There are two arguments that Bob could make:
We get two different results - something is wrong!
Understanding the Confusion
If we don't know what Alice means by "left", the information we get from her is "one of the coins is H". The statement "one of the coins is H" is true if and only if the statement "the configuration is HH or HT or TH" is true. However, note that if Bob hears "the configuration is HH or HT or TH", he would never think to use the first argument. Let's examine the reasoning in more detail:
Assuming that she meant his left with probability 1/2 (thus her left also with probability 1/2), we get:
* She meant his left; the other coin is H (thus H H) - probability 1/4
* She meant his left; the other coin is T (thus H T) - probability 1/4
* She meant her left; the other coin is H (thus H H) - probability 1/4
* She meant her left; the other coin is T (thus T H) - probability 1/4
Now, let's examine the second argument:
When Bob counts the number of outcomes, he is making the assumption that each outcome is equally likely. This seems justified - after all, for fair coins each of the four outcomes is equally likely to begin with.
Notice that we are counting the H H configuration twice in the first case and only once in the second case. This is what leads to the discrepancy, but both arguments seem reasonable. We should be allowed to examine the cases of "his left" and "her left" separately and aggregate the results. We should also be allowed to consider the possible configurations, given that there is H under one of Alice's hands. How come these two arguments lead to different probabilities?
The issue is quite subtle. Hypothetically, consider: what would Alice have said if the coins had landed T T? Maybe she would have said "the left coin is T" or maybe she would have said "both coins landed the same". The point is, Bob doesn't really know how Alice decides what to say - this is the key part of the puzzle. Imagine Alice says "the left coin is H" only for H H and says "the left coin is T" otherwise. If Bob knows this about Alice, then hearing "the left coin is H" would indicate P(H H) = 1.
If Bob knows that Alice picks a hand at random and says what she hides under that hand, this leads to probability 1/2.
If Bob knows that Alice indicates a hand in which she has H whenever she has at least one H, this leads to probability 1/3.
Another way to look at it is that the factual statement "one of the coins is H" does not carry the same information as "Alice said one of the coins is H". The probability Bob assigns to H H depends on what Bob knows about Alice. But maybe Bob just met Alice and knows nothing about her. If you were in Bob's place, how would you come up with a reasonable answer?
Asking Bayes for Help
Let us reason more formally. We will apply Bayesian reasoning to the problem. If we assume all coin toss outcomes are equally probable and then update on evidence, we should get a reasonable answer. Initially, there are 4 hypotheses: HH, HT, TH, TT.
P(HH) = 0.25
P(HT) = 0.25
P(TH) = 0.25
P(TT) = 0.25
We observe the evidence
observation := Alice says "The left coin is Heads."
We need to multiply the probability of each hypothesis by the likelihood of the observation (and normalize).
P(HH) P(observation | HH)
P(HT) P(observation | HT)
P(TH) P(observation | TH)
P(TT) P(observation | TT)
Now is the key question - what is the probability of Alice saying "The left coin is Heads." under the different hypotheses?
These probabilities could be pretty much anything - imagine that Alice is a liar and utters the phrase "The left coin is Heads." when both coins are T and remains silent otherwise. In such case, we would have P(observation | TT) = 1 and all other likelihoods being 0. One more reasonable assumption would be that Alice says what is the coin on Bob's left side. In that case, `P(observation | HH) = P(observation | HT) = 0.5` and `P(observation | TH) = P(observation | TT) = 0`. After multiplying the probabilities by the likelihood and normalizing, we get the posterior probability:
P(HH | observation) = 0.5
P(HT | observation) = 0.5
P(TH | observation) = 0
P(TT | observation) = 0
Note that this is just one possibility for how Alice decides what to say.
Priors for the Observation
We can model Alice as running an algorithm that takes the coin configuration as input and outputs the phrase which Alice utters. The algorithm could be something like this:
def say(configuration):if configuration == "HT":
return "The left coin is Heads."
elif configuration == "TH":
return "The right coin is Heads."
else:
# configuration == "HH" or configuration == "TT"
return "Both coins are the same."
Alice can run any of an infinite number of algorithms
If we assign some prior probability to each algorithm, we can calculate the likelihood (note that the algorithm Alice runs and the coin toss are independent):
The logic is the same for the other hypotheses as well. The first term in the product
can be determined by examining the algorithm
Consider Occam's razor:
In our context we can say that shorter programs are more likely than longer ones. This doesn't give us an answer, it only explains how to approach the problem. In practice, we would need to make some assumptions to get probabilities for the different algorithms ("Alice tells the truth", "Alice always gives relevant information", etc.). In a sense, the problem is underdefined because no prior is specified and we need to assume one.
Many people argue over the Boy or girl paradox, which is basically another variant of what we examine here.
If all this seems a little complicated, it's because Bayesian reasoning is a general framework which can be applied to all reasoning under uncertainty. Humans are complicated and here we need to reason how a human acts. My goal is not to give you the answer (because the problem is underdefined) but to explain where the paradox comes from and the difficulty of the task.