3 Answers sorted by
75
I think you get very different answers depending on whether your question is "what is an example of a policy that makes it illegal in the United States to do research with the explicit intent of creating AGI" or whether it is "what is an example of a policy that results in nobody, including intelligence agencies, doing AI research that could lead to AGI, anywhere in the world".
For the former, something like updates to export administration regulations could maybe make it de-facto illegal to develop AI aimed at the international market. Historically, that was successful at making it illegal to intentionally export software which implemented strong encryption for a bit. It didn't actually prevent the export, but it did arguably make that export unlawful. I'd recommend reading that article in full, actually, to give you an idea of how "what the law says" and "what ends up happening" can diverge.
43
It might be possible to ban a training environment that remains relatively untested? For example, combinations of learning rates or epochs that haven't been documented as safe for achieving an ethical aligned objective. Certainly, implementing such a ban would require a robust global governance mechanism to review what training environment constitutes to achieving an ethically aligned objective but this is how I envision the process of enforcing such a ban could work.
20
State monopoly:
- The Song Dynasty (960-1279) established a state monopoly over saltpeter production, a critical ingredient in gunpowder. The government appointed officials to oversee the collection and refinement of saltpeter.
- During the Ming Dynasty (1368-1644), the government further tightened its control over saltpeter production, with the "Saltpeter Censorate" responsible for managing the state monopoly.
Limiting knowledge:
- Chinese officials kept the recipe for gunpowder a closely guarded secret. The exact proportions of saltpeter, sulfur, and charcoal were not shared widely.
- Technical manuals on gunpowder production, such as the "Wujing Zongyao" (Collection of the Most Important Military Techniques) from the Song Dynasty, were restricted and not freely circulated.
Strict regulations:
- The Ming Dynasty implemented strict laws and regulations on the private manufacture and use of gunpowder weapons. Violators faced severe punishments, including execution.
- In the 14th century, Ming Emperor Hongwu issued an edict prohibiting the private production and sale of gunpowder and firearms, limiting their use to the military.
Emphasis on traditional weapons:
- Confucian scholars and officials promoted the idea that traditional weapons, such as bows and crossbows, were more suitable for maintaining social order and harmony.
- Military examinations during the Ming Dynasty focused on proficiency in traditional weapons rather than gunpowder weapons, reinforcing the importance of traditional warfare techniques.
What's the saltpeter?
You make manufacturing programmable parallel processing hardware, above some certain baseline, a controlled item, and track at all times the location of the chipmaking tooling.
Every IC fab under international regulations would be inspected, and only low power ICs would be permitted without a licence. All our phones, game consoles, computers work as dumb terminals - very little compute is local (computers are all just remote terminals) and as much as possible is in licensed and inspected data centers.
Any internal source codes, documents, etc belonging to AI companies must be seized and classified under the same schema as nuclear secrets.
All software also is a restricted item - being a SWE is a licensed profession, you must have permission from the government (like planning approval) to author anything, and only government licensed software can run on a computer above a certain threshold. The "government software inspectors" will be checking to make sure the implementation isn't just a lazy call to a neural network.
We need to have state controlled media promote the idea that intellectual labor, such as art and computer programming, is more suitable for maintaining social order and harmony. Propaganda must emphasize the danger of trusting any generated assets from AI as soulless cheap copies that will never work.
Schools need to remove any training on using AI from the curriculum - even discussing "prompt engineering" should be a crime.
It wasn't until the the Opium Wars (1839-1842 and 1856-1860) and the Sino-Japanese War (1894-1895) where this ban caused substantial losses to China.
880 years! Getting 20 years out of an AGI ban, before falling to attack from AGI driven weapons from foreign rivals, sounds optimistic.
Just keep in mind the ultimate consequences. Imagine the Chinese soldiers facing machine guns and advanced 19th century warships.
Or:
Prompt: draw a nation that banned AGI facing attack by foreign rivals. Show a battle scene, where all the soldiers in the foreground are old from a lack of regenerative medicine, and the enemy is mostly a sky blotting swarm of drones
[Caveat lector: I know roughly nothing about policy!]
Suppose that there were political support to really halt research that might lead to an unstoppable, unsteerable transfer of control over the lightcone from humans to AGIs. What government policy could exert that political value?
[That does sound relaxing.]
Banning AGI research specifically
This question is NOT ASKING ABOUT GENERALLY SLOWING DOWN AI-RELATED ACTIVITY. The question is specifically about what it could look like to ban (or rather, impose an indefinite moratorium on) research that is aimed at creating artifacts that are more capable in general than humanity.
So "restrict chip exports to China" or "require large vector processing clusters to submit to inspections" or "require evals for commercialized systems" don't answer the question.
The question is NOT LIMITED to policies that would be actually practically enforceable by their letter. Making AGI research illegal would slow it down, even if the ban is physically evadable; researchers generally want to think publishable thoughts, and generally want to plausibly be doing something good or neutral by their society's judgement. If the FBI felt they had a mandate to investigate AGI attempts, even if they would have to figure out some only-sorta-related crime to actually charge, maybe that would also chill AGI research. The question is about making the societal value of "let's not build this for now" be exerted in the most forceful and explicit form that's feasible.
Some sorts of things that would more address the question (in the following, replace "AGI" with "computer programs that learn, perform tasks, or answer questions in full generality", or something else that could go in a government policy):
Why ask this?
I've asked this question of several (5-10) people, some of whom know something about policy and have thought about policies that would decrease AGI X-risk. All of them said they had not thought about this question. I think they mostly viewed it as not a very salient question because there isn't political support for such a ban. Maybe the possibility has been analyzed somewhere that I haven't seen; links?
But I'm still curious because:
Has anyone really been far even as decided to use?
There's a lot of problems with an "AGI ban" policy like this. I'm wondering, though, which problems, if any, are really dealbreakers.
For example, one problem is: How do you even define what "AGI" or "trying to write an AGI" is? I'm wondering how much this is actually a problem, though. As a layman, as far as I know there could be existing government policies that are somewhat comparably difficult to evaluate. Many judicial decisions related to crimes, as I vaguely understand it, depend on intentionality and belief——e.g. for a killing to be a murder, the killer must have intended to kill and must not have believed on reasonable grounds that zer life was imminently unjustifiedly threatened by the victim. So it's not like not-directly-observable mental states are out of bounds. What are some crimes that are defined by mental states that are even more difficult to evaluate? Insider trading? (The problem is still very hairy, because e.g. you have to define "AGI" broadly enough that it includes "generalist scientist tool-AI", even though that phrase gives some plausible deniability like "we're trying to make a thing which is bad at agentic stuff, and only good at thinky stuff". Can you ban "unbounded algorithmic search"?)
Some other comparisons:
Is that good to do?
Yeah, probably, though we couldn't know whether a policy would be good without knowing what the policy would look like. There are some world-destroying things that we have to ban, for now; for everything else, there's
Mastercardlibertarian techno-optimism.