In short: There is no objective way of summarizing a Bayesian update over an event with three outcomes  as an update over two outcomes .
 

Suppose there is an event with possible outcomes .
We have prior beliefs about the outcomes .
An expert reports a likelihood factor of .
Our posterior beliefs about  are then .

But suppose we only care about whether  happens.
Our prior beliefs about  are .
Our posterior beliefs are .
This implies that the likelihood factor of the expert regarding  is .

This likelihood factor depends on the ratio of prior beliefs .

Concretely, the lower factor in the update is the weighted mean of the evidence  and  according to the weights  and .

This has a relatively straightforward interpretation. The update is supposed to be the ratio of the likelihoods under each hypothesis. The upper factor in the update is . The lower factor is .
 

I found this very surprising - the summary of the expert report depends on my prior beliefs!

I claim that this phenomena is unintuitive, and being unaware of this can lead to errors.
 

Why this is weird

Bayes' rule describes how to update our prior beliefs using data.

In my mind, one very nice property of Bayes rule was that it cleanly separates the process into a subjective part (eliciting your priors) and an ~objective part (computing the update).

For example, we may disagree on our prior beliefs on whether eg COVID19 originated in a lab. But we cannot disagree on the direction and magnitude of the update caused by learning that it originated in one of the few cities in the world with a gain-of-function lab working on coronaviruses.

Because of this, researchers are encouraged to report their update factors together with their all considered beliefs. This way, users can use their research for their own conclusions by multiplying their prior with the update. And metastudies can just take the product of the likelihoods of all studies to estimate the combined effect of the evidence.

In the above example, we lose this nice property - the update factor depends on the prior beliefs of the user. Researchers would not be able to objectively summarize their likelihood about whether COVID19 originated in a lab accidentally vs zoonotically vs being designed as a bioweapon as a single number for people who only care about whether it originated in a lab versus any other possibility. 
 

Examples in the wild

I ran into this problem twice recently:

  1. When analyzing Mennen’s ABC example of a case where averaging the logarithmic odds of experts seems to result in nonsense.
  2. In my own research on interpreting Bayesian Networks as I was trying to come up with a way of decomposing a Bayesian update into a combination of several updates.

In both cases being unaware of the phenomena led me to a conceptual mistake.
 

Mennen’s ABC example

Mennen’s example involves three experts debating an event with three possible outcomes, .

Expert #1 assigns relative odds of .
Expert #2 assigns relative odds of .
Expert #3 assigns relative odds of .

The logodds-averaging pooled opinion of the experts is