What to include in a guest lecture on existential risks from AI?

Aryeh Englander

A professor I'm friendly with has been teaching a course on AI ethics this semester, and he asked me if I could come give a guest lecture on "AI apocalypse" scenarios. What should I include in the lecture?

Details:

Audience is mostly graduate engineering students. Some are in a program focused on AI, others are in a systems engineering program and don't otherwise have much knowledge about AI beyond this course.
This will be I think the final class for the semester. Previous classes have covered topics like privacy, bias, explainability, social coordination failures (including race to the bottom dynamics), self-driving cars, AI in social media, lethal autonomous weapons, AI in healthcare, implementing ethics in organizations, and the economic impacts of AI. I did not attend those lectures so I don't know exact details of what was or wasn't covered.
I have leeway to include pretty much anything I want in the class and to focus on any topic(s) I want. The professor would like me to include something about forecasting, but that's optional.
I can optionally assign something short as an assignment before class, but it is the end of the semester so nothing more than that. There will not be any homework based on the class, I think.
The class will be 2 hours long.
I do not have a huge amount of time to prepare for the lecture - it's relatively soon (April 27) and I have multiple other work / school / family obligations I need to attend to between now and then.

If anybody has relevant material I could use, such as slides or activities, that would be great! Also, if anybody wants to help develop the material for this class, please message me (preferably at my work email - Aryeh.Englander@jhuapl.edu).

As a bonus, I expect that material for a class of this sort may turn out to be useful for plenty of other people on this and related forums, either for themselves or as a tool they can use when presenting the same topic to others.

[Note: I am posting this here with permission from the professor.]

Here are the readings/videos I assign when I teach about AI risks to my undergraduates at Smith College:

https://mindstalk.net/vinge/vinge-sing.html

https://iopscience.iop.org/article/10.1088/0031-8949/90/1/018001

Olle Häggstöm had three two hour lectures on AI Safety earlier this spring. Original description was

This six-hour lecture series will treat basics and recent developments in AI risk and long-term AI safety. The lectures are meant to be of interest to Ph.D. students and researchers in AI-related fields, but no particular prerequisites will be assumed.

Lecture 1, Lecture 2 and Lecture 3. Perhaps you can find something there, I expect he would be happy to help if you reach out to him.

any chance you have contact with the people who uploaded that? I suspect the reason I hadn't seen it is that it is marked for being for kids. because of that I can't add it to a playlist. I'm also going to attempt to contact them directly about this.

1Viktor Rehnberg4y

Oh, I hadn't noticed that. I've got some connections to them and can reach out.

Thanks, looks useful!

I have a framing of AI risks scenarios that I think is more general and more powerful than most available online, and that might be a good frame before going into examples. It's not posted yet (I'm finishing the sequence now) but I could sent somethings to you if you're interested. ;)

I would love to add the YouTube video of this class to my database of safety relevant videos once it's out.

copy and pasting channel reviews I wrote originally in my short form - this is too much content to include in a single talk, but I share it in the hope that it will be useful to make the link and perhaps the students would like to see this question itself and discussion around it (I'm a big fan of old fashioned linkweb surfing):

CPAIOR has a number of interesting videos on formal verification, how it works, and some that apply it to machine learning, eg "Safety in AI Systems - SMT-Based Verification of Deep Neural Networks"; "Formal Reasoning Methods in Machine Learning Explainability"; "Reasoning About the Probabilistic Behavior of Classifiers"; "Certified Artificial Intelligence"; "Explaining Machine Learning Predictions"; a few others. https://www.youtube.com/channel/UCUBpU4mSYdIn-QzhORFHcHQ/videos

the collective intelligence workshop from IPAM at UCLA had some recent banger talks on both human and AI network safety: https://www.youtube.com/watch?v=qhjho576fms&list=PLHyI3Fbmv0SfY5Ft43_TbsslNDk93G6jJ

the Schwartz Reisman Institute is a multi-agent safety discussion group, one of the very best ai safety sources I've seen anywhere. a few interesting videos include, for example: "An antidote to Universal Darwinism" - https://www.youtube.com/watch?v=ENpdhwYoF5g

as well as this kickass video on "whose intelligence, whose ethics" https://www.youtube.com/watch?v=ReSbgRSJ4WY

https://www.youtube.com/channel/UCSq8_q4SCU3rYFwnA2bDxyQ

I would also encourage directly mentioning the recent works from Anthropic AI, stuff as this paper from this month, "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback" https://arxiv.org/abs/2204.05862

The simons institute for theoretical computer science at UC Berkeley is a contender for my #1 recommendation from this whole list. Banger talk after banger talk after banger talk there. Several recent workshops with kickass ai safety focus. https://www.youtube.com/user/SimonsInstitute

A notable recent workshop is "learning in the presence of strategic behavior": https://www.youtube.com/watch?v=6Uq1VeB4h3w&list=PLgKuh-lKre101UQlQu5mKDjXDmH7uQ_4T

another fun one is "learning and games": https://www.youtube.com/watch?v=hkh23K3-EKw&list=PLgKuh-lKre13FSdUuEerIxW9zgzsa9GK9

they have a number of "boot camp" lessons that appear to be meant for an interdisciplinary advanced audience as well. the current focus of talks is on causality and games, and they also have some banger talks on "how not to run a forecasting competition", "the invisible hand of prediction", "communicating with anecdotes", "the challenge of understanding what users want", and my personal favorite due to its fundamental reframing of what game theory even is, "in praise of game dynamics": https://www.youtube.com/watch?v=lCDy7XcZsSI

In general I have a higher error rate than some folks on less wrong and my recommendations should be considered weaker and more exploratory. but here you go, those are my exploratory recommendations, and I have lots and lots more suggestions for more capability focused stuff on my short form.

I would love to add the YouTube video of this class to my database of safety relevant videos once it's out.

CPAIOR has a number of interesting videos on formal verification, how it works, and some that apply it to machine learning, eg "Safety in AI Systems - SMT-Based Verification of Deep Neural Networks"; "Formal Reasoning Methods in Machine Learning Explainability"; "Reasoning About the Probabilistic Behavior of Classifiers"; "Certified Artificial Intelligence"; "Explaining Machine Learning Predictions"; a few others. https://www.youtube.com/channel/UCUBpU4mSYdIn-QzhORFHcHQ/videos

the collective intelligence workshop from IPAM at UCLA had some recent banger talks on both human and AI network safety: https://www.youtube.com/watch?v=qhjho576fms&list=PLHyI3Fbmv0SfY5Ft43_TbsslNDk93G6jJ

the Schwartz Reisman Institute is a multi-agent safety discussion group, one of the very best ai safety sources I've seen anywhere. a few interesting videos include, for example: "An antidote to Universal Darwinism" - https://www.youtube.com/watch?v=ENpdhwYoF5g

as well as this kickass video on "whose intelligence, whose ethics" https://www.youtube.com/watch?v=ReSbgRSJ4WY

https://www.youtube.com/channel/UCSq8_q4SCU3rYFwnA2bDxyQ

I would also encourage directly mentioning the recent works from Anthropic AI, stuff as this paper from this month, "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback" https://arxiv.org/abs/2204.05862

The simons institute for theoretical computer science at UC Berkeley is a contender for my #1 recommendation from this whole list. Banger talk after banger talk after banger talk there. Several recent workshops with kickass ai safety focus. https://www.youtube.com/user/SimonsInstitute

A notable recent workshop is "learning in the presence of strategic behavior": https://www.youtube.com/watch?v=6Uq1VeB4h3w&list=PLgKuh-lKre101UQlQu5mKDjXDmH7uQ_4T

another fun one is "learning and games": https://www.youtube.com/watch?v=hkh23K3-EKw&list=PLgKuh-lKre13FSdUuEerIxW9zgzsa9GK9

they have a number of "boot camp" lessons that appear to be meant for an interdisciplinary advanced audience as well. the current focus of talks is on causality and games, and they also have some banger talks on "how not to run a forecasting competition", "the invisible hand of prediction", "communicating with anecdotes", "the challenge of understanding what users want", and my personal favorite due to its fundamental reframing of what game theory even is, "in praise of game dynamics": https://www.youtube.com/watch?v=lCDy7XcZsSI

20

[ Question ]

What to include in a guest lecture on existential risks from AI?

20

Ω 10

20

Ω 10

3 Answers sorted by
top scoring

Apr 13, 2022

Apr 13, 2022

Apr 13, 2022

20

Ω 10

20

[ Question ]

What to include in a guest lecture on existential risks from AI?

20

Ω 10

20

Ω 10

3 Answers sorted by top scoring

Apr 13, 2022

Apr 13, 2022

Apr 13, 2022

20

Ω 10

3 Answers sorted by
top scoring