# Is the sum individual informativeness of two independent variables no more than their joint informativeness?

by Brangus 3mo8th Jul 20193 comments

# 11

Is it true that:

If I(X;Y) = 0 then I(S;X) + I(S;Y) <= I(S;X,Y)

Can you find a counterexample, or prove this and teach me your proof?

Someone showed me a simple analytic proof. I am still interested in seeing different ways people might prove this though.

New Comment
Write here. Select text for formatting options.
We support LaTeX: Cmd-4 for inline, Cmd-M for block-level (Ctrl on Windows).
You can switch between rich text and markdown in your user settings.

jessicata

### Jul 08, 2019

12

For a visualization, see information diagrams, and note that the central cell I(S; X; Y) must be non-positive (because I(S; X; Y) + I(X; Y | S) = I(X; Y) = 0).

Miss_Figg

### Jul 08, 2019

3

We want to prove:

This can be rewritten as:

After moving everything to the right hand side and simplifying, we get:

Now if we just prove that is a probability distribution, then the left hand side is , and Kullback-Leibler divergence is always nonnegative.

Ok, q is obviously nonnegative, and its integral equals 1:

Q.e.d.