AI strategy research projects project generators prompts

Mostly for personal use. Likely to be expanded over time.

Some prompts inspired by Framing AI strategy:

• Plans
  • What plans would be good?
  • Given a particular plan that is likely to be implemented, what interventions or desiderata complement that plan (by making it more likely to succeed or by being better in worlds where it succeeds)
• Affordances: for various relevant actors, what strategically significant actions could they take? What levers do they have? What would it be great for them to do (or avoid)?
• Intermediate goals: what goals or desiderata are instrumentally useful?
• Threat modeling: for various threats, model them well enough to understand necessary and sufficient conditions for preventing them.
• Memes (& frames): what would it be good if people believed or paid attention to?

For forecasting prompts, see List of uncertainties about the future of AI.

Some miscellaneous prompts:

• Slowing AI
  • How can various relevant actors slow AI?
  • How can the AI safety community slow AI?
  • What considerations or side effects are relevant to slowing AI?
• How do labs act, as a function of [whatever determines that]? In particular, what's the deal with "racing"?
• AI risk advocacy
  • How could the AI safety community do AI risk advocacy well?
  • What considerations or side effects are relevant to AI risk advocacy? 
• What's the deal with crunch time?
• How will the strategic landscape be different in the future?
• What will be different near the end, and what interventions or actions will that enable? In particular, is eleventh-hour coordination possible? (Also maybe emergency brakes that don't appear until the end.)
• What concrete asks should we have for labs? for government?
• Meta: how can you help yourself or others do better AI strategy research?

The dataset is public and includes a question "how long have you worked in" the "AI research area [you have] worked in for the longest time," so you could check something related!

Yeah, these seem like useful concepts in some contexts too.

I don't understand this sentence:

"Post-AGI compute overhang" here describes the gap between compute used to build AGI in the first place vs. more efficient designs that AI-aided progress could quickly discover.

It's the gap between the training compute of 'the first AGI' and what?

I largely agree. But I think not-stacking is only slightly bad because I think the "crappy toy model [where] every alignment-visionary's vision would ultimately succeed, but only after 30 years of study along their particular path" is importantly wrong; I think many new visions have a decent chance of succeeding more quickly and if we pursue enough different visions we get a good chance of at least one paying off quickly.

Edit: even if alignment researchers could stack into just a couple paths, I think we might well still choose to go wide.

Combating bad regulation would be the obvious way.

In seriousness, I haven’t focused on interventions to improve regulation yet— I just noticed a thing about public opinion and wrote it. (And again, some possible regulations would be good.)

But please consider, are you calling for regulation because it actually makes sense, or because it's the Approved Answer to problems?

I didn't call for regulation.

Some possible regulations would be good and some would be bad.

I do endorse trying to nudge regulation to be better than the default.

I expect to update this comment with additional sources—and perhaps new analytic frames—as I become aware of them and they become public. Last updated 21 March 2023.

Affordances:

• [draft] Matthijs Maas's "Levers of governance" in "Transformative AI Governance: A Literature Review"
• Eugene Bardach's Things Governments Do (affordances for states from a non-AI perspective) (thanks to Matthijs Maas for this suggestion)
• Observation: you get different taxonomies if you start at goals (like "slow China") vs levers (like "immigration policy"). And your uncertainties are like "how can actor X achieve goal Y" vs "how can actor X leverage its ability Z." Maybe you think of different affordances; try both ways.
• Alex Gray mentions as a motivating/illustrative example (my paraphrasing): windfall clauses (or equity-sharing or other benefit-sharing mechanisms) are unlikely to be created by labs but it's relatively easy for labs to take an existing windfall-clause affordance

Intermediate goals:

• Rethink Priorities's Survey on intermediate goals in AI governance
• [draft] Matthijs Maas's "Parameters of Transformative AI Governance"

Theories of victory:

• Matthijs Maas's Strategic Perspectives on Transformative AI Governance (2022) and related work in progress

Memes & frames:

• Maybe Holden Karnofsky's Spreading messages to help with the most important century

Leverage:

• [draft] Alex Lintz's "A simple model for how to prioritize different timelines"

If I was rewriting this post today, I would probably discuss something like object-level frames or strategic perspectives. They make aspects of a situation more salient; whether or not they’re true, and whether or not they’re the kind-of-thing that can be true, they can be useful. See Matthijs Maas's Strategic Perspectives on Transformative AI Governance for illustration.

This largely feels true.

But if someone is disposed to believe P because of strong argument X, the existence of weak arguments for X doesn't feel like it dissuades them.

There's a related phenomenon where--separate from what people believe and how they evaluate arguments--your adversaries will draw attention to the most objectionable things you say, and a movement's adversaries will draw attention to the most objectionable things a member of the movement says.

I roughly support slowing AI progress (although the space of possibilities has way more dimensions than just slow vs fast). Some takes on "Reasons one might try to accelerate progress":

• Avoid/delay a race with China + Keep the good guys in the lead. Sure, if you think you can differentially accelerate better actors, that's worth noticing. (And maybe long timelines means more actors in general, which seems bad on net.) I feel pretty uncertain about the magnitude of these factors, though.
• Smooth out takeoff. Sure, but be careful-- this factor suggests faster progress is good insofar as it's due to greater spending. This is consistent with trying to slow timelines by e.g. trying to get labs to publish less.
• Another factor is non-AI x-risk: if human-level AI solves other risks, and greater exposure to other risks doesn't help with AI, this is a force in favor of rolling the dice on AI sooner. (I roughly believe non-AI x-risk is much smaller than the increase in x-risk from shorter timelines, but I'm flagging this as cruxy; if I came to believe that e.g. biorisk was much bigger, I would support accelerating AI.)

Interesting, thanks.

(I agree in part, but (1) planning for far/slow worlds is still useful and (2) I meant more like metrics or model evaluations are part of an intervention, e.g. incorporated into safety standards than metrics inform what we try to do.)