A prompt reply! Wonderful. Thank you. Apologies for the bold, it would indeed have been more proper in any academic context to append "(emphasis mine)". A crass error.
I wouldn't endorse the last sentence, but I don't think that's a crux for your criticism.
An agreeable amendment to the post, since that line was copied from it (including the '-ish'):
The better solution is what I did in the second model: To create a finer categorization of the space of things that might be true, such that the probability of the data is constant-ish for each term. (emphasis mine)
Either you apply the procedure with this stopping rule for when to stop discretizing, or it boils down to "discretize until the subjective prior and likelihood lead to agreement with what you imagine the posterior intuitively ought to look like", which seems empty; one may as well state the subjective posterior without rationalizing it, all the prior is doing is adding discretization error (you indeed do state that just eliciting the posterior directly is not cogent, but if the procedure only runs on intuition, then I don't know that this is any different).
I believe your Dutch Book argument is equivalent to the statement that the above procedure is not decision-theoretic optimal.
No. Suboptimal is "you could have done better", of course you could. Dominated is "you lose something for free every state." It may be true that the theoretical full prior is preferable to the coarse prior; the data-dependent in-between is incoherent, and an easy rectification is available which does not lead to such problems. This is not a claim about optimality.
The only decision-theoretic optimal procedure is to state your full prior (without any approximation) and make decisions based on that.
No. Proceeding with the coarse prior is coherent and undominated, as are averaging, stacking, nonparametrics. Also coherent: data splitting. Etc. It is sometimes true that it is too expensive or inconvenient to be fully, coherently Bayesian; typically when this is argued some binding argument is provided that things will still 'work out okay' somehow (say, logical induction, or most proper approximations generally), showing that one is being appropriately approximately Bayesian. I do not believe there is such a bound here, it seems to me as though this produces arbitrary answers in general (through careful picking of level-sets, say). It is sometimes hard to notice that by loosening one of the Bayesian screws one has been led to something that produces arbitrary, uncheckable answers, but this seems like one of those cases.
if you follow my procedure but only make the discretization more fine that is literally an example of model expansion
No. Model 1 gives a different likelihood ratio for the data when compared to Model 2 (9:1 vs. 1:1). Model expansion qua Gelman means the model agrees with the simpler model as a special case; that's what makes it different from just changing the model. If you marginalize the finer model here, you recover a different model. Indeed, it is possible to do model expansion coherently, while this is not coherent, thus it is not that.
The comparison is fruitful in what the post does not recommend: when Gelman recommends expansion as a sensible way to make inferences from data, there are a trillion little sense checks he suggests so that one doesn't fool themselves into modelling towards a foregone conclusion (prior and posterior predictive checks, prior sensitivity analysis, there's a whole diagram), and indeed he is constantly careful in making sure not to give yourself too many 'researcher degrees of freedom', to have a coherent workflow where each step is checkable and has sense. Of course, you don't need to explain the whole thing, but sufficient warning that this is dangerous territory seems at least appropriate if not just correct.
This is closer to 'model criticism', which he makes no qualms about separating from the properly, fully Bayesian part of an analysis. Though it still isn't that, because there's a methodical aspect to it which (something like) "the posterior looks implausible" does not capture.
There is a place in the world for posts targeting a general audience, so I can't accept this criticism.
Fair enough. Still, "this is one particular way to address this well-known problem; others are A, B or C" seems non-technical and cogent. When presenting one method among many, it seems best to present weaknesses as well as strengths. If you observe the other comments, it seems as though some other commenters seem to believe that this is perfectly Bayesian and in little to no tension with the formalism. I believe this to be a reasonable reading of what was written, and that something else should have been written instead.
Not only am I not claiming that this is the only way to deal with the general problem, I'm not even claiming that it's the only way to deal with this specific example. I am stating that the the only way to choose a better discretization for this example is to look at the data.
I don't know. Is that true? If you imagine that the prior is fixed, then that the likelihood is fixed, then the posterior it produces is the posterior it produces.
If the probabilities are for betting, these are plainly worse than the coarse ones. If they are meant to be more accurate, in the Bayesian sense, they are not - see the accuracy theorems. If they are for convincing, I don't know that they convince anyone - it seems to me like they might be able to help you convince yourself that your answer is Bayesian in some sense that it is not, but I don't know that you can in any sense argue persuasively against someone who simply dismisses the post-data model out of hand as post-hoc and ad-hoc. Maybe you can state the 'rules' more plainly? You seemed to agree with my description and that one allows any conclusion (really, you disagreed with the only part that could have disallowed it).
If we are allowed to complain about the man's prior, a simple predictive check would have sufficed (his opinion does not change after seeing the data, so even before seeing anything the model may be coherently rejected). To be frank I am not convinced that it is impossible to consider "what if some witnesses are unreliable" without necessarily seeing the data. Then, whichever refinement one produces is perfectly coherent and possibly much better, insofar as it incorporates whichever other relevant prior information may work out.
All I've claimed is that there are often perfectly good reason to change your prior after looking at the data. And as far as I can tell, you accept that this is true. So what am I missing? How do I square this with the very broad and very sharp criticisms you made in your first comment?
There are "often good reasons for changing the prior after seeing the data", yes. The problem is that your suggestion is that it should be fine to do this in an ad hoc way until the posterior either reaches that arbitrary stopping point (with the near-constant likelihood) or until the Bayes rule is made to agree with intuition (which seems to be the stopping rule in the example?). Just because I concede "it is sometimes good to surgically open a man up" does not mean I agree to clefting the poor Bayesian in twain.
Other procedures preserve coherence (as above) perfectly well (or require a psychic diachronic bookie, unlike this one), a third type have either efficiency or robustness guarantees that make the model modification at least somewhat convincing. With your method, I expect an aliens-believer to roll his eyes and produce his own bisection that accords with his principles.
My initial comment was fairly vague, yes. As I said, it is a general fact that such a data-dependent prior produces incoherency, so I thought that the existence statement would be enough to convince one that maybe this has a pretty hefty (and avoidable) cost. I am glad to see that producing the counterexample instead of saying it exists seems to have helped communicate the sense in which this is bad. And, fair enough, I was incautious in saying that you abide by any generic double-counting.
On the quote being out of context: a more general claim appears before any aliens are in mention (for real problems, I've come to believe that refusing to change your prior after you see the data often leads to tragedy). But at the end, somehow, the thesis stops referring to "real problems" and only to "aliens". The sentence is the post's last line, following "So". The idea is that you meant to say that this refers not only to specifically the toy "aliens" example, but also specifically in picking a discretization? If that is truly the intention, I humbly ask that you consider appending your post with "at least for picking a discretization in this aliens example". I don't think I am alone in supposing you meant to say more than this.
They say you’re supposed to choose your prior in advance. That’s why it’s called a “prior”. First, you’re supposed to say say how plausible different things are, and then you update your beliefs based on what you see in the world.
For example, currently you are—I assume—trying to decide if you should stop reading this post and do something else with your life. If you’ve read this blog before, then lurking somewhere in your mind is some prior for how often my posts are good. For the sake of argument, let’s say you think 25% of my posts are funny and insightful and 75% are boring and worthless.
OK. But now here you are reading these words. If they seem bad/good, then that raises the odds that this particular post is worthless/non-worthless. For the sake of argument again, say you find these words mildly promising, meaning that a good post is 1.5× more likely than a worthless post to contain words with this level of quality.
If you combine those two assumptions, that implies that the probability that this particular post is good is 33.3%. That’s true because the red rectangle below has half the area of the blue one, and thus the probability that this post is good should be half the probability that it’s bad (33.3% vs. 66.6%)
(Why half the area? Because the red rectangle is ⅓ as wide and ³⁄₂ as tall as the blue one and ⅓ × ³⁄₂ = ½. If you only trust equations, click here for equations.)(Why half the area? Because the red rectangle is ⅓ as wide and ³⁄₂ as tall as the blue one and ⅓ × ³⁄₂ = ½. If you only trust equations, click here for equations.)
It’s easiest to calculate the ratio of the odds that the post is good versus bad, namely
It follows that
and thus that
Alternatively, if you insist on using Bayes’ equation:
Theoretically, when you chose your prior that 25% of dynomight posts are good, that was supposed to reflect all the information you encountered in life before reading this post. Changing that number based on information contained in this post wouldn’t make any sense, because that information is supposed to be reflected in the second step when you choose your likelihood
p[good | words]. Changing your prior based on this post would amount to “double-counting”.In theory, that’s right. It’s also right in practice for the above example, and for the similar cute little examples you find in textbooks.
But for real problems, I’ve come to believe that refusing to change your prior after you see the data often leads to tragedy. The reason is that in real problems, things are rarely just “good” or “bad”, “true” or “false”. Instead, truth comes in an infinite number of varieties. And you often can’t predict which of these varieties matter until after you’ve seen the data.
Aliens
Let me show you what I mean. Say you’re wondering if there are aliens on Earth. As far as we know, there’s no reason aliens shouldn’t have emerged out of the random swirling of molecules on some other planet, developed a technological civilization, built spaceships, and shown up here. So it seems reasonable to choose a prior it’s equally plausible that there are aliens or that there are not, i.e. that
Meanwhile, here on our actual world, we have lots of weird alien-esque evidence, like the Gimbal video, the Go Fast video, the FLIR1 video, the Wow! signal, government reports on unidentified aerial phenomena, and lots of pilots that report seeing “tic-tacs” fly around in physically impossible ways. Call all that stuff
data. If aliens weren’t here, then it seems hard to explain all that stuff. So it seems likeP[data | no aliens]should be some low number.On the other hand, if aliens were here, then why don’t we ever get a good image? Why are there endless confusing reports and rumors and grainy videos, but never a single clear close-up high-resolution video, and never any alien debris found by some random person on the ground? That also seems hard to explain if aliens were here. So I think
P[data | aliens]should also be some low number. For the sake of simplicity, let’s call it a wash and assume thatSince neither the prior nor the data see any difference between aliens and no-aliens, the posterior probability is
See the problem?
(Click here for math.)
Observe that
where the last line follows from the fact that
P[aliens] ≈ P[no aliens]andP[data | aliens] ≈ P[data | no aliens]. Thus we have thatWe’re friends. We respect each other. So let’s not argue about if my starting assumptions are good. They’re my assumptions. I like them. And yet the final conclusion seems insane to me. What went wrong?
Assuming I didn’t screw up the math (I didn’t), the obvious explanation is that I’m experiencing cognitive dissonance as a result of a poor decision on my part to adopt a set of mutually contradictory beliefs. Say you claim that Alice is taller than Bob and Bob is taller than Carlos, but you deny that Alice is taller than Carlos. If so, that would mean that you’re confused, not that you’ve discovered some interesting paradox.
Perhaps if I believe that
P[aliens] ≈ P[no aliens]and thatP[data | aliens] ≈ P[data | no aliens], then I must accept thatP[aliens | data] ≈ P[no aliens | data]. Maybe rejecting that conclusion just means I have some personal issues I need to work on.I deny that explanation. I deny it! Or, at least, I deny that’s it’s most helpful way to think about this situation. To see why, let’s build a second model.
More aliens
Here’s a trivial observation that turns out to be important: “There are aliens” isn’t a single thing. There could be furry aliens, slimy aliens, aliens that like synthwave music, etc. When I stated my prior, I could have given different probabilities to each of those cases. But if I had, it wouldn’t have changed anything, because there’s no reason to think that furry vs. slimy aliens would have any difference in their eagerness to travel to ape-planets and fly around in physically impossible tic-tacs.
But suppose I had divided up the state of the world into these four possibilities:
No aliens + normal peopleNo aliens + weird peopleNormal aliensWeird aliensIf I had broken things down that way, I might have chosen this prior:
Now, let’s think about the empirical evidence again. It’s incompatible with
no aliens + normal people, since if there were no aliens, then normal people wouldn’t hallucinate flying tic-tacs. The evidence is also incompatible withnormal alienssince is those kinds of aliens were around they would make their existence obvious. However, the evidence fits pretty well withweird aliensand also withno aliens + weird people.So, a reasonable model would be
If we combine those assumptions, now we only get a 10% posterior probability of aliens.
Now the results seem non-insane.
(math)
(math)
To see why, first note that
since both
normal aliensandno aliens + normal peoplehave near-zero probability of producing the observed data.Meanwhile,
where the second equality follows from the fact that the data is assumed to be equally likely under
no aliens + weird peopleandweird peopleIt follows that
and so
Huh?
I hope you are now confused. If not, let me lay out what’s strange: The priors for the two above models both say that there’s a 50% chance of aliens. The first prior wasn’t wrong, it was just less detailed than the second one.
That’s weird, because the second prior seemed to lead to completely different predictions. If a prior is non-wrong and the math is non-wrong, shouldn’t your answers be non-wrong? What the hell?
The simple explanation is that I’ve been lying to you a little bit. Take any situation where you’re trying to determine the truth of anything. Then there’s some space of things that could be true.
In some cases, this space is finite. If you’ve got a single tritium atom and you wait a year, either the atom decays or it doesn’t. But in most cases, there’s a large or infinite space of possibilities. Instead of you just being “sick” or “not sick”, you could be “high temperature but in good spirits” or “seems fine except won’t stop eating onions”.
(Usually the space of things that could be true isn’t easy to map to a small 1-D interval. I’m drawing like that for the sake of visualization, but really you should think of it as some high-dimensional space, or even an infinite dimensional space.)
In the case of aliens, the space of things that could be true might include, “There are lots of slimy aliens and a small number of furry aliens and the slimy aliens are really shy and the furry aliens are afraid of squirrels.” So, in principle, what you should do is divide up the space of things that might be true into tons of extremely detailed things and give a probability to each.
Often, the space of things that could be true is infinite. So theoretically, if you really want to do things by the book, what you should really do is specify how plausible each of those (infinite) possibilities is.
After you’ve done that, you can look at the data. For each thing that could be true, you need to think about the probability of the data. Since there’s an infinite number of things that could be true, that’s an infinite number of probabilities you need to specify. You could picture it as some curve like this:
(That’s a generic curve, not one for aliens.)
To me, this is the most underrated problem with applying Bayesian reasoning to complex real-world situations: In practice, there are an infinite number of things that can be true. It’s a lot of work to specify prior probabilities for an infinite number of things. And it’s also a lot of work to specify the likelihood of your data given an infinite number of things.
So what do we do in practice? We simplify, usually by limiting creating grouping the space of things that could be true into some small number of discrete categories. For the above curve, you might break things down into these four equally-plausible possibilities.
Then you might estimate these data probabilities for each of those possibilities.
Then you could put those together to get this posterior:
That’s not bad. But it is just an approximation. Your “real” posterior probabilities correspond to these areas:
That approximation was pretty good. But the reason it was good is that we started out with a good discretization of the space of things that might be true: One where the likelihood of the data didn’t vary too much for the different possibilities inside of
A,B,C, andD. Imagine the likelihood of the data—if you were able to think about all the infinite possibilities one by one—looked like this:This is dangerous. The problem is that you can’t actually think about all those infinite possibilities. When you think about four four discrete possibilities, you might estimate some likelihood that looks like this:
If you did that, that would lead to you underestimating the probability of
A,B, andC, and overestimating the probability ofD.This is where my first model of aliens went wrong. My prior
P[aliens]was not wrong. (Not to me.) The mistake was in assigning the same value toP[data | aliens]andP[data | no aliens]. Sure, I think the probability of all our alien-esque data is equally likely given aliens and given no-aliens. But that’s only true for certain kinds of aliens, and certain kinds of no-aliens. And my prior for those kinds of aliens is much lower than for those kinds of non-aliens.Technically, the fix to the first model is simple: Make
P[data | aliens]lower. But the reason it’s lower is that I have additional prior information that I forgot to include in my original prior. If I just assert thatP[data | aliens]is much lower thanP[data | no aliens]then the whole formal Bayesian thing isn’t actually doing very much—I might as well just state that I thinkP[aliens | data]is low. If I want to formally justify whyP[data | aliens]should be lower, that requires a messy recursive procedure where I sort of add that missing prior information and then integrate it out when computing the data likelihood.(math)
Mathematically,
But now I have to give a detailed prior anyway. So what was the point of starting with a simple one?
I don’t think that technical fix is very good. While it’s technically correct (har-har) it’s very unintuitive. The better solution is what I did in the second model: To create a finer categorization of the space of things that might be true, such that the probability of the data is constant-ish for each term.
The thing is: Such a categorization depends on the data. Without seeing the actual data in our world, I would never have predicted that we would have so many pilots that report seeing tic-tacs. So I would never have predicted that I should have categories that are based on how much people might hallucinate evidence or how much aliens like to mess with us. So the only practical way to get good results is to first look at the data to figure out what categories are important, and then to ask yourself how likely you would have said those categories were, if you hadn’t yet seen any of the evidence.