Forethought^[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.

We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.

More details on our website.

Why we exist

We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared.

Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future.

Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization.

Research

Research agendas

We are currently pursuing the following perspectives:

Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destruction; what rights to give digital beings; how to govern an automated military; and how to avoid dictatorship or authoritarianism.
Achieving a near-best future: Most explicitly longtermist work to date has been focused on avoiding existential catastrophe, but achieving a near-best future might be even more important. Research avenues here include mapping out what a desirable “viatopia” would look like (i.e. a state of the world which is very likely to lead to a very good future), figuring out how space resources should be allocated, and addressing issues around AI epistemics and persuasion.

We tend to think that many non-alignment areas of work are particularly neglected.

However, we are not confident that these are the best frames for this work, and we are keen to work with people who are pursuing their own agendas.

Recent work

Today we’re also launching “Preparing for the Intelligence Explosion”, which ...

Posts

Wikitag Contributions

Comments