This is the second of two meetups that KWR is hosting jointly with EA Waterloo. We'll be meeting in room 1506 of the QNC Building in the University of Waterloo campus at 7pm.

Please note: for KWR attendees, this is half an hour earlier than the usual starting time.

For this meetup, we'll be discussing AI alignment. From the LW tag description for AI:

AI Alignment is the task of ensuring [powerful] AI system are aligned with human values and interests. The central concern is that a powerful enough AI, if not designed and implemented with sufficient understanding, would optimize something unintended by its creators and pose an existential threat to the future of humanity. This is known as the AI alignment problem.

A study of machine learning researchers in 2022 indicated that they believe that the probability of the long-run effect of advanced AI on humanity being “extremely bad (e.g., human extinction)” is 5-10%. I think this is a pretty strong signal that this is something we need to pay attention to, if nothing else.

At the meetup, we'll discuss:

• Should we be worried?
• If we're not worried right now, at what AI development milestones do we need to start paying attention?
• If people are interested in discussing more items, put them in the comments here!

Readings

Here are the suggested readings; please complete them before the meetup on Thursday:

Pause Giant AI Experiments: An Open Letter (Future of Life Institute, 22 March 2023)
https://futureoflife.org/open-letter/pause-giant-ai-experiments/

Pausing AI Developments Isn't Enough. We Need to Shut it All Down (Eliezer Yudkowsky, 29 March 2023)
https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/

Nobody's on the ball on AGI Alignment (leopold on EA forums, 29 March 2023)
https://forum.effectivealtruism.org/posts/5LNxeWFdoynvgZeik/nobody-s-on-the-ball-on-agi-alignment

Optional Supplementary Readings

For those that have very little AI background but want to participate, here is an2 part post on how AI works, in plain English and with no math:

How AI works, plain English, no math (Jon Evans, November 2022)
https://aiascendant.substack.com/p/how-dall-e-works-part-1

And here is FLI's in depth policy recommendations for the 6 month pause:

Policymaking in the Pause (Future of Life Institute, 19 April 2023) https://futureoflife.org/wp-content/uploads/2023/04/FLI_Policymaking_In_The_Pause.pdf