List of uncertainties about the future of AI

This is an unordered list of uncertainties about the future of AI, trying to be comprehensive– trying to include everything reasonably decision-relevant and important/tractable.

This list is mostly written from a forecasting perspective. A useful complementary perspective to forecasting would be strategy or affordances or what actors can do and what they should do. This list is also written from a nontechnical perspective.

• Timelines
  • Capabilities as a function of inputs (or input requirements for AI of a particular capability level)
  • Spending by leading labs
  • Cost of compute
  • Ideas and algorithmic progress
  • Endogeneity in AI capabilities
  • What would be good? Interventions?
• Takeoff (speed and dynamics)
  • dcapabilities/dinputs (or returns on cognitive reinvestment or intelligence explosion or fast recursive self-improvement): Is there a threshold of capabilities such that self-improvement or other progress is much greater slightly above that point than slightly below it? (If so, where is it?) Will there be a system that can quickly and cheaply improve itself (or create a more capable successor), such that the improvements enable similarly large improvements, and so on until the system is much more capable? Will a small increase in inputs cause a large increase in capabilities (like the difference between chimps and humans) (and if so, around human-level capabilities or where)? How fast will progress be?
    • Will dcapabilities/dinputs be very high because of recursive self-improvement?
    • Will dcapabilities/dinputs be very high because of generality being important and monolithic/threshold-y?
    • Will dcapabilities/dinputs be very high because ideas are discrete (and in particular, will there be a single "secret sauce" idea)? [Seems intractable and unlikely to be very important.]
  • dimpacts/dcapabilities: Will small increases in capabilities (on the order of the variance between different humans) cause large increases in impacts?
    • Related: payoff thresholds and human-competition threshold
  • Qualitatively, how will AI research capabilities affect AI progress; what will AI progress look like when AI research capabilities are a big deal?
    • One dimension or implication: will takeoff be local or nonlocal? How distributed will it be; will it look like recursive self-improvement or the industrial revolution?
      • What does this depend on?
  • Endogeneity in AI capabilities: how do AI capabilities affect AI capabilities? (Potential alternative framing: dinputs/dtime.)
    • How (much) will AI tools accelerate research?
    • Will AI labs generate substantial revenue?
    • Will AI systems make AI seem more exciting or frightening? What effect would this have on AI progress?
    • What other endogeneities exist?
• Weak AI
  • How will the strategic landscape be different in the future?
    • Due to weak AI?
    • Due to other factors? (See Relevant pre-AGI possibilities.)
  • Will weak AI make AI risk clearer, and especially make it more legible (relates to "warning shots")?
  • Will AI progress cause substantial misuse or conflict? What would that look like?
  • Will AI progress enable pivotal acts or processes?
• Misalignment risk sources/modes
  • What misalignment would occur by default, and how would it be bad? Possible (overlapping) scenarios include inner alignment failure, outer alignment failure, getting what you measure, influence-seeking, more Paul Christiano stories, multipolar failure, a treacherous turn, a sharp left turn (or distributional leap), and more.
  • What would a powerful, unaligned AI agent do? [Answer: the details are unpredictable, but the high-level outcome is pretty clear; it would very likely be catastrophic. (But note there is reasonable disagreement, e.g..)]
• Technical problems around AI systems doing what their controllers want [doesn't fit into this list well]
  • How to make powerful AI systems aligned to human preferences
    • Or: how to make powerful AI systems that do not cause a catastrophe
  • How to make powerful AI systems interpretable to human understanding
  • How to solve problems around decision theory and strategic interaction (and whether they're important)
  • How to solve Wei-Dai-style philosophy problems (and whether they're important)
  • How to solve problems around delegation involving multiple humans or multiple AI systems (and whether they're important)
• Polarity (relates to takeoff, endogeneity, and timelines)
  • What determines or affects polarity?
  • What are the effects or implications of polarity on alignment and stabilization?
  • What are the effects or implications of polarity on what the long-term future looks like, conditional on achieving alignment and stabilization?
  • What would be good? Interventions?
• Proximate and ultimate uses of powerful AI
  • What uses of powerful AI would be great? How good would various possible uses of powerful AI be?
  • Conditional on achieving alignment, what's likely to occur (and what could foresighted actors cause to occur or not to occur)?
• Agents vs tools and general systems vs narrow tools (relates to tool AI and Comprehensive AI Services)
  • Are general systems more powerful than similar narrow tools?
  • Are agents more powerful than similar tools?
  • Does generality appear by default in capable systems? (This relates to takeoff.)
  • Does agency (or goal-directedness or consequentialism or farsightedness) appear by default in capable systems?
• AI labs' behavior and racing for AI
  • How will labs think about AI?
  • What actions could labs perform; what are they likely to do by default?
  • What would it be better if labs did; what interventions are tractable?
• States' behavior
  • How will states think about AI?
  • What actions could states perform? What are they likely to do by default?
  • What would it be better if states did; what interventions are tractable?
• Public opinion
  • How the public thinks about AI and framing; what the public thinks about AI, what memes would spread widely and how that depends on other facts about the world; how all of that translates into attitudes and policy preferences
  • Wakeup to capabilities
  • Wakeup to alignment risk and warning shots for alignment
  • What (facts about public opinion) would be good? Interventions?
• Paths to powerful AI (relates to timelines, AI risk modes, and more)
  • How successful will reinforcement-learning agents built on large language models be?
  • How successful will comprehensive AI services be?
  • How successful will language model bureaucracies be?
  • How successful will STEM AI or Skunkworks-style AI be?
  • How successful will simulating evolution be?
  • How successful will whole-brain emulation or neuromorphic AI be?
  • When is thinking in terms of paths or roadmaps useful? What other high-level paths are relevant?
• Meta and miscellanea
  • Epistemic stuff
    • Research methodology and organization: how to do research and organize researchers
    • Forecasting methodology: how to do forecasting better
    • Collective epistemology: how to share and aggregate beliefs and knowledge
  • Decision theory
  • Simulation hypothesis
    • Do we live in a simulation? (If so, what's the deal?)
    • What should we do if we live in a simulation (as a function of what the deal is)?
  • Movement/community/field stuff

Maybe this list would be more useful if it had more pointers to relevant work?

Maybe this list would be more useful if it included stuff that's important that I don't feel uncertain about? But probably not much of that exists?

I like lists/trees/graphs. I like the ideas behind Clarifying some key hypotheses in AI alignment and Modelling Transformative AI Risks. Perhaps this list is part of the beginning of a tree/graph for AI forecasting not including alignment stuff.