Awesome, really glad to hear it was helpful, thanks for commenting!

Yep, fixed, thanks!

Or "prompting" ? Seems short and memorable, not used in many other contexts so its meaning would become clear, and it fits in with other technical terms that people are currently using in news articles, e.g. "prompt engineering". (Admittedly though, it might be a bit premature to guess what language people will use!)

This is awesome, I love it! Thanks for sharing (-:

Thank you :-)

Thanks, really appreciate it!

I think some of the responses here do a pretty good job of this. It's not really what I intended to go into with my post since I was trying to keep it brief (although I agree this seems like it would be useful).

And yeah, despite a whole 16 lecture course on convex opti I still don't really get Bregman either, I skipped the exam questions on it 😆

Oh yeah, I hadn't considered that one. I think it's interesting, but the intuitions are better in the opposite direction, i.e. you can build on good intuitions for $D_{KL}$ to better understand MI. I'm not sure if you can easily get intuitions to point in the other direction (i.e. from MI to $D_{KL}$), because this particular expression has MI as an expectation over $D_{KL}$, rather than the other way around. E.g. I don't think this expression illuminates the nonsymmetry of $D_{KL}$.

The way it's written here seems more illuminating (not sure if that's the one that you meant). This gets across the idea that:

$P_{(X,Y)}$ is the true reality, and $P_X\otimes P_Y$ is our (possibly incorrect) model which assumes independence. The mutual information between $X$ and $Y$ equals $D_{KL}(P_{(X,Y)}||P_X\otimes P_Y)$, i.e. the extent to which modelling $X$ and $Y$ as independent (sharing no information) is a poor way of modelling the true state of affairs (where they do share information). 

But again I think this intuition works better in the other direction, since it builds on intuitions for $D_{KL}$ to better explain MI. The arguments in the $D_{KL}$ expression aren't arbitrary (i.e. we aren't working with $D_{KL}\left(P||Q\right)$), which restricts the amount this can tell us about $D_{KL}$ in general.