Let be a variable in $P$ for the true hypothesis, and let $H_{k}$ be the possible values of $H,$ such that $H_{k}$ is mutually exclusive and exhaustive. Then, Bayes' theorem states:

$P (H_{i} ∣ e) = \frac{P (e ∣ H_{i}) \cdot P (H_{i})}{\sum_{k} P (e ∣ H_{k}) \cdot P (H_{k})},$

with a proof that runs as follows. By the definition of conditional probability,

$P (H_{i} ∣ e) = \frac{P (e \land H_{i})}{P (e)} = \frac{P (e ∣ H_{i}) \cdot P (H_{i})}{P (e)}$

By the law of marginal probability:

$P (e) = \sum k P (e \land H_{k})$

By the definition of conditional probability again:

$P (e \land H_{k}) = P (e ∣ H_{k}) \cdot P (H_{k})$

Done.

Note that this proof of Bayes' rule is less general than the proof of the odds form of Bayes' rule.

Example

Using the Diseasitis example problem, this proof runs as follows:

$\begin{matrix} P (s i c k ∣ p o s i t i v e) = \frac{P (p o s i t i v e \land s i c k)}{P (p o s i t i v e)} = \frac{P (p o s i t i v e \land s i c k)}{P (p o s i t i v e \land s i c k) + P (p o s i t i v e \land \neg s i c k)} = \frac{P (p o s i t i v e ∣ s i c k) \cdot P (s i c k)}{(P (p o s i t i v e ∣ s i c k) \cdot P (s i c k)) + (P (p o s i t i v e ∣ \neg s i c k) \cdot P (\neg s i c k))} \end{matrix}$

Numerically:

$3 / 7 = \frac{0.18}{0.42} = \frac{0.18}{0.18 + 0.24} = \frac{90 % * 20 %}{(90 % * 20 %) + (30 % * 80 %)}$

Using red for sick, blue for healthy, and + signs for positive test results, the proof above can be visually depicted as follows:

bayes theorem probability

if we replace the other Venn diagram for the proof of Bayes' rule, we should probably update this one too.

Proof of Bayes' rule: Probability form