This workshop will address the technical and institutional questions of how to safeguard human interests after AI surpasses human abilities.

This workshop will build on many domains, including political theory, cooperative AI, economics, mechanism design, history, and hierarchical agency.  We hope to gather insights about the roles humans could play in a world transformed by AGI, and which positive equilibria are stable, if any.

Speakers so far:

• Joe Carlsmith, Open Philanthropy - keynote on "Can goodness compete?"
• Jacob Steinhardt, Stanford & Transluce
• Richard Ngo
• Anna Yelizarova, Windfall
• Stephen Casper, MIT & UK AISI
• Emmett Shear, Softmax
• Beatrice Erkers, Foresight Institute (tentative)
• Fazl Barez, Oxford AI Governance Initiative
• Rif A. Saurous, Google

Discussion will center on empirical questions such as:

• Could alignment of single AIs to single humans be sufficient to solve global coordination problems?
• Will agency tend to operate at ever-larger scales, multiple scales, or something else?
• Are there multiple, qualitatively different basins of attraction of future civilizations?
• Do Malthusian conditions necessarily make it hard to preserve uncompetitive, idiosyncratic values?
• What empirical evidence could help us tell which trajectory we’re on?

For more information, see the event website or fill out the expression of interest form.

Don't wait for the workshop - give your takes on these questions here too!