Call for suggestions - AI safety course

Boaz Barak

In the fall I am planning to teach an AI safety graduate course at Harvard. The format is likely to be roughly similar to my "foundations of deep learning" course.

I am still not sure of the content, and would be happy to get suggestions.

Some (somewhat conflicting) desiderata:

I would like to cover the various ways AI could go wrong: malfunction, misuse, societal upheaval, arms race, surveillance, bias, misalignment, loss of control,... (and anything else I'm not thinking of right now). I talke about some of these issues here and here.
I would like to see what we can learn from other fields, including software security, aviation and automative safety, drug safety, nuclear arms control, etc.. (and happy to get other suggestions)
Talk about policy as well, various frameworks inside companies, regulations etc..
Talk about predictions for the future, methodologies for how to come up with them.
All of the above said, I get antsy if I don't get my dosage of math and code- I intend 80% of the course to be technical and cover research papers and results. It should also involve some hands on projects.
Some technical components should include: evaluations, technical mitigations, attacks, white and black box interpretability methods, model organisms.
Whenever I teach a course I always like to learn something from it, so I hope to cover state of art research results, especially ones that require some work to dig into and I wouldn't get to do so without this excuse.
Anyway, since I haven't yet planned this course, I thought I would solicit comments on what should be covered in such a course. Links to other courses blogs etc. are also useful. (I do have a quirk that I've never been able to teach from someone else's material, and often ended up writing a textbook, see here, here, and here whenever I teach a course.., so I don't intent to adapt any curricula wholesale)

Thanks in advance!

In the fall I am planning to teach an AI safety graduate course at Harvard. The format is likely to be roughly similar to my "foundations of deep learning" course.

I am still not sure of the content, and would be happy to get suggestions.

Some (somewhat conflicting) desiderata:

I would like to cover the various ways AI could go wrong: malfunction, misuse, societal upheaval, arms race, surveillance, bias, misalignment, loss of control,... (and anything else I'm not thinking of right now). I talke about some of these issues here and here.
I would like to see what we can learn from other fields, including software security, aviation and automative safety, drug safety, nuclear arms control, etc.. (and happy to get other suggestions)
Talk about policy as well, various frameworks inside companies, regulations etc..
Talk about predictions for the future, methodologies for how to come up with them.
All of the above said, I get antsy if I don't get my dosage of math and code- I intend 80% of the course to be technical and cover research papers and results. It should also involve some hands on projects.
Some technical components should include: evaluations, technical mitigations, attacks, white and black box interpretability methods, model organisms.
Whenever I teach a course I always like to learn something from it, so I hope to cover state of art research results, especially ones that require some work to dig into and I wouldn't get to do so without this excuse.
Anyway, since I haven't yet planned this course, I thought I would solicit comments on what should be covered in such a course. Links to other courses blogs etc. are also useful. (I do have a quirk that I've never been able to teach from someone else's material, and often ended up writing a textbook, see here, here, and here whenever I teach a course.., so I don't intent to adapt any curricula wholesale)

Thanks in advance!

If legal policy is in your wheelhouse, here's a selection of the growing literature (apologies, some of it is my own)

Noam Kolt – "Algorithmic Black Swans"
Addressing catastrophic tail events via anticipatory regulation
Published: https://journals.library.wustl.edu/lawreview/article/id/8906/
Peter Salib & Simon Goldstein – "AI Rights for Human Safety"
On AI rights for safety
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4536494
Yonathan Arbel, Matthew Tokson & Albert Lin – "Systemic Regulation of Artificial Intelligence"
Moving from application-level regulation to system-level regulation
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4543681
Published: https://arizonastatelawjournal.org/article/systemic-regulation-of-artificial-intelligence/
Gabriel Weil – “Tort Law as a Tool for Mitigating Catastrophic Risk from Artificial Intelligence” Proposes reforming tort law to deter catastrophic AI risks before they materialize
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4694006
Yonathan Arbel et al. – "Open Questions in Law and AI Safety: An Emerging Research Agenda"
Sets out a research agenda for a new field of “AI Safety Law,” focusing on existential and systemic AI risks
Published: https://www.lawfaremedia.org/article/open-questions-in-law-and-ai-safety-an-emerging-research-agenda
Peter Salib – "AI Outputs Are Not Protected Speech"
Argues that AI-generated outputs lack First Amendment protection, enabling stronger safety regulation
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4481512
Mirit Eyal & Yonathan Arbel – “Tax Levers for a Safer AI Future”
Proposes using tax credits and penalties to align AI development incentives with public safety
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4528105
Cullen O’Keefe, Rohan Ramakrishnan, Annette Zimmermann, Daniel Tay & David C. Winter – “Law-Following AI: Designing AI Agents to Obey Human Laws”
Suggests AI agents should be trained and constrained to follow human law, like corporate actors
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4726207

54

Call for suggestions - AI safety course

54

54

54

Call for suggestions - AI safety course

54

54