Many AI safety researchers these days are not aiming for a full solution to AI safety (e.g., the classic Friendly AI), but just trying to find good enough partial solutions that would buy time for or otherwise help improve global coordination on AI research (which in turn would buy more time for AI safety work), or trying to obtain partial solutions that would only make a difference if the world had a higher level of global coordination than it does today.
My question is, who is thinking directly about how to achieve such coordination (aside from FHI's Center for the Governance of AI, which I'm aware of) and where are they talking about it? I personally have a bunch of questions related to this topic (see below) and I'm not sure what's a good place to ask them. If there's not an existing online forum, it seems a good idea to start thinking about building one (which could perhaps be modeled after the AI Alignment Forum, or follow some other model).
- What are the implications of the current US-China trade war?
- Human coordination ability seems within an order of magnitude of what's needed for AI safety. Why the coincidence? (Why isn’t it much higher or lower?)
- When humans made advances in coordination ability in the past, how was that accomplished? What are the best places to apply leverage today?
- Information technology has massively increased certain kinds of coordination (e.g., email, eBay, Facebook, Uber), but at the international relations level, IT seems to have made very little impact. Why?
- Certain kinds of AI safety work could seemingly make global coordination harder, by reducing perceived risks or increasing perceived gains from non-cooperation. Is this a realistic concern?
- What are the best intellectual tools for thinking about this stuff? Just study massive amounts of history and let one's brain's learning algorithms build what models it can?
I have since been given access to a sample of such non-public discussions. (The sample is small but I think at least somewhat representative.) Worryingly, it seems that there's a disconnect between the kind of global coordination that AI governance researchers are thinking and talking about, and the kind that technical AI safety researchers often talk about nowadays as necessary to ensure safety.
In short, the Google docs I've seen all seem to assume that a safe and competitive AGI can be achieved at some reasonable level of investment into technical safety, and the main coordination problem is how to prevent a "race to the bottom" whereby some actors try to obtain a lead in AI capabilities by underinvesting in safety. However, current discussion among technical AI safety researchers suggest that a safe and competitive AGI perhaps can't be achieved at any feasible level of investment into technical safety, and at a certain point we'll probably need global coordination to stop, limit, or slow down progress in and/or deployment/use of AI capabilities.
Questions I'm trying to answer now: 1) Is my impression from the limited sample correct? 2) If so, how best to correct this communications gap (and prevent similar gaps in the future) between the two groups of people working on AI risk?
I appreciate how you turned the most useful private info into public conversation while largely minimising the amount of private info that had to become public.
To respond directly, yes, your observation matches my impression of folks working on governance issues who aren’t very involved in technical alignment (with the exception of Bostrom). I have no simple answer to the latter question.
I have a feeling it's not that simple. See the last part of “Generate evidence of difficulty” as a research purpose on biases. So for example I know at least one person who quit from an AI safety org (in part) because they became convinced that it's too difficult to achieve safe, competitive AI (or at least the approach pursued by the org wasn't going to work). Another person privately told me they have little idea how their research will eventually contribute to a safe, competitive AI, but hasn't written anything like that publicly AFAIK. (And note that I don't actually have that many opportunities to speak privately with other AI safety researchers.) Another thing is that most AI safety researchers probably don't think it's part of their job to "generate evidence of difficulty" so I have to convince them of that first.
Unless these problems are solved, I might be able to convince a few safety researchers to go to governance researchers and tell them they think it's not possible to get safe, competitive AI, but their concerns will probably just be dismissed as outliers. I think a better step forward would be to build a private forum where these kinds of concerns can be more frankly discussed, as well as a culture where doing so is normative. This addresses some of the possible biases and I'm still not sure about the others.