This is a linkpost for http://squirrelinhell.blogspot.com/2017/09/understanding-policy-gradients.html
From what I've read so far, I think Information Theory, Inference and Learning Algorithms does a rather good job of conveying the intuitions behind topics.