A breakdown of priors and posteriors - an example from medicine

by masasin3 min read1st Oct 20184 comments

8

Frontpage

This post is lifted (with edits) from my comment on a thread on Reddit that talked about a blood test that detected amyloid plaque, a potential precursor to Alzheimers. A top-level commenter felt that the fact that the test couldn't tell for certain whether someone had Alzheimers meant that it was useless.

"Not everyone with amyloid in their brains will turn out to have dementia, and not everyone who has dementia will be found to have amyloid in their brains."

So what is the point of this article again? To get a load of clicks I assume, cos there's no real content here.

I attempted to correct that misconception.


tl;dr: No test is ever 100% certain, but it can still constrain your probabilities, making you more certain than you were before. Not everyone who is outside during a thunderstorm gets hit by lightning, and not everyone who gets hit by lightning was outside during a thunderstorm. Going outside in a thunderstorm is still much more dangerous than staying inside.


Notation: is read as the probability of having dementia given that you have amyloid. To make things fit in a LessWrong post, I'll abbreviate having dementia to , and having amyloid to . A positive test result is , and a negative test result is . Finally, a negation is , so would be no amyloid.

, read as the odds of dementia to no dementia, is the odds ratio that is true compared to the odds ratio that is false. means that it's 3 times as likely that somebody has dementia than that they don't. In general, odds doesn't say anything about the magnitude of the probabilities, so they could be small, like 3% and 1%, or big, like 60% and 20%. (Here, of course, because the two choices represent all possibilities, so the probabilities have to be 75% and 25% respectively.)


Imagine you have ten thousand people in the right demographics. How about these (the numbers are made up):

Dementia status Has amyloid Does not have amyloid Total
Has dementia 1000 300 1300
Does not have dementia 100 8600 8700
Total 1100 8900 10000

So, a person chosen at random from that population has a chance of having dementia, and a chance of not having dementia. Their odds ratio for dementia is . They're almost 7 times as likely to have no dementia than they are to have it.

Now, let's say that if you have amyloid, the test will say that you have amyloid 95% of the time, while if you do not have amyloid, the test will say that you have amyloid 10% of the time. In other words, , and . The probability of negative results is just 100% minus the probability of the corresponding positive result. (The actual status is on the top row, and the test response is on the left.)

Test result Has amyloid Does not have amyloid
Positive test 95% 10%
Negative test 5% 90%

Now, let's say that you take the test and it says that you do have amyloid. What can we say about your probability of dementia? Can we do better than that 13%?

If the test is positive, one of four things might be true:

  • The test is correct, and you do have amyloid, . You're one of those people who has dementia and amyloid, .
  • The test is correct, and you do have amyloid, . However, you're one of those people who has amyloid despite not having dementia, .
  • The test is incorrect, and you do not have amyloid, . However, you're one of those people who has dementia despite not having amyloid, .
  • The test is incorrect, and you do not have amyloid, . You're one of the majority who have neither dementia nor amyloid, .

Now, we can find the probability that you do have dementia if you get a positive test result:

We can also find the probability that you have dementia if you get a negative test result:

So, we end up with one final table. The probability that someone doesn't have dementia given a test result is just 100% minus the probability of having dementia. (What the test says is on top, and whether you actually have dementia is on the left.)

Dementia status Positive test Negative test
Has dementia 82.6% 8.0%
Does not have dementia 17.4% 92.0%

In other words, a random person would have a 13% chance of having dementia, . However, if your test is positive for amyloid, we adjust to an 82.6% chance that you have dementia, , and a 17.4% chance that you don't, . This is considering the fact that some people have amyloid but no dementia, the fact that some people have no amyloid and dementia, and the fact that the test can just be wrong.

On the other hand, if the test is negative, it would go down from 13% to 8%, but that still isn't zero. It could be that you were just unlucky and the test didn't register your amyloid.

If the test is positive, you're almost 5 times more certain that you have dementia than that you don't have dementia, since . You are now 32 times more sure that you have dementia than you were before. ()

If the test is negative, you're more than 11 times more certain that you don't have dementia than that you do, since . Compared to your prior belief, you're almost twice as sure that you don't have dementia than you were before. () (Notice how we flipped the fractions since we're testing the negative case.)

I wouldn't consider a negative case that big of a deal, but the positive one certainly warrants further testing. If you use another test, which checks for things other than amyloid, or for amyloid using a different mechanism, you'll be much more sure of your result because your starting probability would now be the 82.6%, not the original 13%. (The order of the tests doesn't matter, but a first screening would probably be with a test that is cheap and/or quick.)


edit: Fixed the explanation of odds ratios. Thanks to Joachim Bartosik for pointing it out.

Frontpage

8