Mammography problem from 'Intro to Bayes' in my own words/picture

[-]Scott Alexander15y260

I got to shadow some breast physicians last month, and although it's sort of off topic I think I gained some insight as to why so many doctors get this question wrong.

Which is because it's very different from any situation they ever come across in clinical practice. Guidelines are to screen people with mammography and examination; anyone who comes up as suspicious on those two tests then gets a biopsy. No one gets diagnosed with breast cancer from a mammogram alone, the progression from mammogram on to the next step is hard-coded into a pre-determined algorithm, and so the question of "This woman got a positive on the mammogram; does she have cancer?" never comes up. A question that does come up a lot is a woman panicking because she got a positive mammogram and demanding to know whether she has breast cancer, and the inevitable answer is "We'll need to do more tests, but don't worry too much yet because most of these things are false positives."

So the doctors involved know that most real mammogram results are false positives, they know how to diagnose breast cancer based on the combination of tests they actually do, they just can't do Bayesian math problems when given probabilities. This is kind of interesting if you're curious about their intelligence but as far as I know doesn't really affect clinical care.

[-]jake98772215y50

As far as the take-home practical message goes, on my reading it was never about how well doctors could "diagnose cancer" per se based on mammogram results--rather, the reason we ask about P(cancer | positive) is because it ought to inform our decision about whether a biopsy is really warranted. If a healthy young woman from a population with an exceedingly low base rate for breast cancer has a positive mammogram, the prior probability of her having cancer may still be low enough that there might actually be negative expected value in following up with a biopsy; after all, let's not forgot that a biopsy is not a trivial procedure and things do sometimes go wrong.

So I think this actually does have some implication for real-world clinical care: we ought to question whether it is wise to automatically follow up all positive mammograms with biopsies. Maybe it is, and maybe it isn't, but I don't think we should take the question for granted as appears to be the case.

[-]Alicorn15y90

If a biopsy is the next step in diagnosing breast cancer after a positive mammogram, then we shouldn't perform mammograms on anyone it still wouldn't be worth biopsying should their mammogram turn up positive.

[-]jake98772215y10

Yes, that's exactly right.

And although I'm having a hard time finding a news article to verify this, someone informed me that the official breast cancer screening recommendations in the US (or was it a particular state, perhaps California?) were recently modified such that it is now not recommended that women younger than 40 (50?) receive regular screening. The young woman who informed me of this change in policy was quite upset about it. It didn't make any sense to her. I tried to explain to her how it actually made good sense when you think about it in terms of base rates and expected values, but of course, it was no use.

But to return to the issue clinical implications, yes: if a woman belongs to a population where the result of a mammogram would not change our decision about whether a biopsy is necessary, then probably she shouldn't have the mammogram. I suspect that this line of reasoning would sound quite foreign to most practicing doctors.

[-]jwhendy15y00

Now that's some interesting back story. I could see that if one knew that the route from positive test through to establishing conclusively if cancer is present was a fluid path, one might not be overly concerned with the test result itself.

As to this specific problem, I just used what EY used; perhaps there are more applicable/pertinent statistics problems that could be used.

I've been trying to think of other visualization tools that might be more universal or intuitive. I get the sliding java applets, but think if one can tie what they're showing to a real world tool or process of some sort, it will help. What these are doing is no different than his. The "top bar" are the two original spheres. The "bottom bar" is the reduced amount of each sphere (0.8 x 0.01 and 0.096 x 0.99) that remains after sifting.

Just a different way to look at it.

[-]Kevin15y40

Thanks for the post, but I think this Venn Diagram style visualization of the problem is way more intuitive.

http://oscarbonilla.com/2009/05/visualizing-bayes-theorem/

[-]jwhendy15y10

Absolutely, and thanks for the link. That was fantastic. Using a Venn diagram occurred to me, but I hadn't worked out exactly how to use it.

The one thing I will say about this is that I like the idea that using something (e.g. a test) tangible on the individuals or objects contributes to the idea that the new evidence is adding something to the pre-existing knowledge. The Venn diagram shows quite clearly what the final result was, but I like the idea that you're gathering more information, which contributes to the body of knowledge, and that this is what alone allows you to update/slide your estimate.

In the Venn diagram, they just start drawing circles and that's that.

In this one, it's a little more clear (at least to me) that the test is doing different things depending on who is being tested and that this is what creates the end inequality between cancer and non-cancer positive tests.

Does that make sense?

Thanks again for sharing. Honestly, I don't see these visualizations as one size fits all; the more who contribute or popularize the various tools out there, the better everyone is!

[-]Subsumed15y20

I really like this sieve approach. I feel a big improvement would be to show the output of the sieve as two boxes (red and blue) as well to help emphasize visually just how many false+ pass through and the relative size of false+ to all that pass through.

[-]jwhendy15y00

Check the update. I'm not quite sure how to describe visually what's "left" in the sieve. I don't want to show both test+ and false+ as outputs, exactly, because a sieve is suppose to keep some stuff back while letting other stuff through. But, I think the idea of the circles above makes things more clear than the bars in terms of what's going on as well as proportionality.

I do cover the equivalent of fasle+ verbally, but it would be nice to make it visually. I'll keep thinking about this. Part of it is that I was trying to set things up onto just one page. If I ditch that (it's already on two now), I could maybe spread things out even more and show what's left in the sieves for each group.

Thanks for the suggestion.

[-]hairyfigment15y20

Hmm, this seems like a good approach, but I don't know that I'd understand the graphic if I didn't already know what it meant.

[-]jwhendy15y00

Yeah, I can definitely see that. I thought about two different sieves and more of a "splitting' of the groups, but tried to illustrate that the same test is being used on everyone, not different tests depending on whether you have cancer or not.

Maybe "l'll try that version and see if it's clearer.

[-]Swimmer963 (Miranda Dixon-Luinenburg)15y10

Good idea! Images always make things clearer for me. Your redo is very nicely done, too.

[-]jwhendy15y00

Thanks. In the end, I agree that the Venn diagram is much "cleaner." I actually thought of using one at one point, I guess it's been done!

LESSWRONG
LW

LESSWRONG
LW

5

Mammography problem from 'Intro to Bayes' in my own words/picture

5

5