A course I tought on AI safety and alignment at Princeton from 2 years ago:
https://sites.google.com/view/cos598aisafety/

list of materials is on the webpage, happy to share lecture slides as well. 

I believe Peter Henderson has a newer version. Roger Grosse taught a similar course at U Toronto.