Hi Alex,

I suspect that you might find the following articles interesting:

1. Information-Theoretic Probing with Minimum Description Length;
2. Evaluating representations by the complexity of learning low-loss predictors;
3. A Theory of Usable Information Under Computational Constraints;
4. A Novel Approach to the Partial Information Decomposition.