[ Question ]

Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?

by elityre3 min read29th Jul 201927 comments


Coordination / Cooperation

(Or, is coordination easier in a long timeline?)

It seems like it would be good if the world could coordinate to not build AGI. That is, at some point in the future, when some number of teams will have the technical ability to build and deploy and AGI, but they all agree to voluntarily delay (perhaps on penalty of sanctions) until they’re confident that humanity knows how to align such a system.

Currently, this kind of coordination seems like a pretty implausible state of affairs. But I want to know if it seems like it becomes more or less plausible as time passes.

The following is my initial thinking in this area. I don’t know the relative importance of the factors that I listed, and there’s lots that I don’t understand about each of them. I would be glad for…

  • Additional relevant factors.
  • Arguments that some factor is much more important than the others.
  • Corrections, clarifications, or counterarguments to any of this.
  • Other answers to the question, that ignore my thoughts entirely.

If coordination gets harder overtime, that’s probably because...

  • Compute increases make developing and/or running an AGI cheaper. The most obvious consideration is that the cost of computing falls each year. If one of the bottlenecks for an AGI project is having large amounts of compute, then “having access to sufficient compute” is a gatekeeper criterion on who can build AGI. As the cost of computing continues to fall, more groups will be able to run AGI projects. The more people who can build an AGI, the harder it becomes to coordinate all of them into not deploying it.
    • Note that It is unclear to what degree there is currently, or will be, a hardware overhang. If someone in 2019 could already run an AGI, on only $10,000 worth of AWS, if only they knew how, then the cost of compute is not relevant to the question of coordination.
  • The number of relevant actors increases. If someone builds an AGI in the next year, I am reasonably confident that that someone will be Deep Mind. I expect that in 15 years, if I knew that AGI would be developed one year from then, it will be much less overdetermined which group is going to build it, because there will be many more well funded AI teams with top talent, and, most likely, none of them will have as strong a lead as Deep Mind currently appears to have.
    • This consideration suggests that coordination gets harder over time. However, this depends heavily on other factors (like how accepted AI safety memes are) that determine how easily Deep Mind could coordinate internally.

If coordination gets easier over time, that’s probably because…

  • AI safety memes become more and more pervasive and generally accepted. It seems that coordination is easier in worlds where it is uncontroversial and common knowledge that an unaligned AGI poses and existential risk, because everyone agrees that they will lose big if anyone builds an AGI.
    • Over the past 15 years, the key arguments of AI safety have gone from being extremely fringe, to a reasonably regarded (if somewhat controversial) position, well inside the overton window. Will this process continue? Will it be commonly accepted by ML researches in 2030, that advanced AI poses and existential threat? Will it be commonly accepted by the leaders of nation-states?
    • What will the perception of safety be in a world where there is another AGI winter? Suppose that narrow ML proves to be extremely useful in a large number of fields, but there’s lots of hype about AGI being right around the corner, then that bubble bursts, and there is broad disinterest in AGI again. What happens to the perception of AI safety? Is there a sense of “It looks like AI Alignment wasn’t important after all”? How cautious will researchers be in developing new AI technologies.
  • [Partial subpoint to the above consideration] Individual AI teams develop more serious info security conscious processes. If some team in Deep Mind discovered AGI today, and the Deep Mind leadership opted to wait to insure safety before deploying it, I don’t know how long it would be until some relevant employees left to build AGI on their own, or some other group (such as a state actor) stole their technology and deployed it.
    • I don’t know if this is getting better or worse, overtime.
  • The technologies for maintaining surveillance of would-be AGI developers improve. Coordination is made easier by technologies that aid in enforcement. If surveillance technology improves that seems like it would make coordination easier. As a special case, highly reliable lie detection or mind reading technologies would be a game-changer for making coordination easier.
    • Is there a reason to think that offense will beat defense in this area? Surveillance could get harder over time if the technology for detecting and defeating surveillance outpaces the technology for surveilling.
  • Security technology improves. Similarly, improvements in computer security (and traditional info security), would make it easier for actors to voluntarily delay deploying advanced AI technologies, because they could trust that their competitors (other companies and other nations), wouldn’t be able to steal their work.
    • I don’t know if this is plausible at all. My impression is that the weak point of all security systems is the people involved. What sort of advancements would make the human part of a security system more reliable?


New Answer
Ask Related Question
New Comment

5 Answers

A missing point in favor of coordination getting easier: AI safety as a field seems likely to mature over time, and as it does the argument "let's postpone running this AGI code until we first solve x" may become more compelling, as x increases in legibility and tractability.

I suspect that one of the factors that will make coordinating to not build AGI harder is that the incentive to build AGI will become greater for a larger amount of people. Right now, there's a large class of people who view AI as a benign technology, that will bring about large amounts of economic growth, and that it's effects are going to be widespread and positive. I think this position is best captured by Andew Ng when he says "AI is the new electricity". Likewise, the Whitehouse states "Artificial intelligence holds the promise of great benefits for American workers, with the potential to improve safety, increase productivity, and create new industries we can’t yet imagine.".

However, as time goes by AI capabilities grow and so will public demonstrations of what's possible with AI. This will cause people to revise upwards their beliefs about the impact/power of AI and AGI and drag far more actors into the game. I think that if the Whitehouse shared the views of DeepMind or OpenAI on AGI, they wouldn't hesitate to start the equivalent of a second Manhattan project.

elityre makes a sincere effort to examination of the question from the ground up. But this overlooks the work that's already been done in similar fields. A lot of what has been accomplished with regard to applied genetic research is likely to be transferable, for instance.

More generally, formal methods of safety engineering can provide a useful framework, when adapted flexibly to reflect novel aspects of the question.

New consideration: hyperbolic time discounting suggests it gets harder over time. It's easier to lose a benefit that seems far off in the future than to lose a benefit that seems imminent.

(Though usually I think of this consideration as suggesting that coordination right now will be easier than we think.)

The question makes the assumption that "The World" is in any way coordinating or will ever coordinate to build AGI. I posit that "The World" has not and will not coordinate anything.

Are there global corporations and super power governments that are coordinating AI projects? Yes, but its not a singular AI project, The AI-sphere contains multitudes.

Also, AGI is a specific term and although its become more popular, its mostly a term Goertzel created because the term "AI" was being improperly used to label even simplistic statistical models like deep learning networks. At least that is how I saw it when I first read the term. I'm still looking for a free copy of AGI Revolution.