LESSWRONG
LW

Wikitags

Proof of Bayes' rule: Probability form

Edited by So8res, Eliezer Yudkowsky, et al. last updated 9th Oct 2016
Requires: Bayes' rule, Bayes' rule: Probability form, Conditional probability, Probability, Math 2
Teaches: Bayes' rule, Bayes' rule: Probability form, Proof of Bayes' rule

Let H be a variable in P for the true hypothesis, and let Hk be the possible values of H, such that Hk is mutually exclusive and exhaustive. Then, Bayes' theorem states:

P(Hi∣e)=P(e∣Hi)⋅P(Hi)∑kP(e∣Hk)⋅P(Hk),

with a proof that runs as follows. By the definition of conditional probability,

P(Hi∣e)=P(e∧Hi)P(e)=P(e∣Hi)⋅P(Hi)P(e)

By the law of marginal probability:

P(e)=∑kP(e∧Hk)

By the definition of conditional probability again:

P(e∧Hk)=P(e∣Hk)⋅P(Hk)

Done.

Note that this proof of Bayes' rule is less general than the proof of the odds form of Bayes' rule.

Example

Using the Diseasitis example problem, this proof runs as follows:

P(sick∣positive)=P(positive∧sick)P(positive)=P(positive∧sick)P(positive∧sick)+P(positive∧¬sick)=P(positive∣sick)⋅P(sick)(P(positive∣sick)⋅P(sick))+(P(positive∣¬sick)⋅P(¬sick))

Numerically:

3/7=0.180.42=0.180.18+0.24=90%∗20%(90%∗20%)+(30%∗80%)

Using red for sick, blue for healthy, and + signs for positive test results, the proof above can be visually depicted as follows:

bayes theorem probability

Parents:
Proof of Bayes' rule
Bayes' rule: Probability form
3
3
Discussion0
Discussion0