User Comment Replies

We want to prove:

\int \int p_{x s} (x, s) log \frac{p_{x s} (x, s)}{p_{s} (s) p_{x} (x)} d x d s + \int \int p_{y s} (y, s) log \frac{p_{y s} (y, s)}{p_{s} (s) p_{y} (y)} d y d s \leq \int \int \int p_{x y s} (x, y, s) log \frac{p_{x y s} (x, y, s)}{p_{s} (s) p_{x y} (x, y)} d x d y d s

This can be rewritten as:

\int \int \int p_{x y s} (x, y, s) log \frac{p_{s x} (s, x)}{p_{s} (s) p_{x} (x)} d x d y d s + \int \int \int p_{x y s} (x, y, s) log \frac{p_{s y} (s, y)}{p_{s} (s) p_{y} (y)} d x d y d s \leq \int \int \int p_{x y s} (x, y, s) log \frac{p_{x y s} (x, y, s)}{p_{s} (s) p_{x} (x) p_{y} (y)} d x d y d s

After moving everything to the right hand side and simplifying, we get:

\int \int \int p_{x y s} (x, y, s) log \frac{p_{x y s} (x, y, s) p_{s} (s)}{p_{x s} (x, s) p_{y s} (y, s)} d x d y d s \geq 0

Now if we just prove that $q (x, y, s) = \frac{p_{x s} (x, s) p_{y s} (y, s)}{p_{s} (s)}$ is a probability distribution, then the left hand side is $K L (p_{x y s} (x, y, s), q)$ , and K... (read more)

LESSWRONG
LW

All of Miss_Figg's Comments + Replies