I recently worked with Andrea Miotti on a draft of a treaty that would implement global compute caps. I'm including the abstract from our paper and the treaty text below. 

Aside: I'm interested in follow-up work that aims to strengthen the treaty, describe compute limitations in more detail, and discuss alternative paths toward global compute caps. There is a growing community of people who are interested in developing, strengthening, and advocating for global compute caps and related proposals. If you're interested in this, please feel free to reach out. 

Abstract

This paper presents an international treaty to reduce risks from the development of advanced artificial intelligence (AI). The main provision of the treaty is a global compute cap: a ban on the development of AI systems above an agreed-upon computational resource threshold. The treaty also proposes the development and testing of emergency response plans, negotiations to establish an international agency to enforce the treaty, the establishment of new communication channels and whistleblower protections, and a commitment to avoid an AI arms race. We hope this treaty serves as a useful template for global leaders as they implement governance regimes to protect civilization from the dangers of advanced artificial intelligence.

Treaty

TREATY ON THE PROHIBITION OF DANGEROUS ARTIFICIAL INTELLIGENCE

The States Parties to this Treaty,

Deeply concerned about the catastrophic consequences that would be visited upon all humankind by a disaster induced by advanced artificial intelligence,

Acknowledging the need to make every effort to avert the danger of such a catastrophe and to take measures to safeguard international peace and security,

Affirming that artificial intelligence poses risks at least as severe as those from nuclear war, uncontrolled pandemics, and other major threats to global security,

Believing that the creation of human-level artificial intelligence or artificial superintelligence should only occur once the international community is confident that such technologies can be controlled and that the necessary national and international governance measures have been established,

Recognizing that global security risks from artificial intelligence can occur either from uncontrolled artificial intelligence systems or from human misuse,

Determined to eliminate and prevent artificial intelligence race dynamics between countries and between corporations which significantly raise the risk of catastrophe,

Acknowledging the benefits that advanced artificial intelligence could bring to humanity once there is greater certainty that such technology can be developed and governed safely,

Expressing their support for research, development, and other efforts to safeguard the production and trade of powerful artificial intelligence hardware and to identify privacy-preserving methods of monitoring compliance with hardware regulations,

Reaffirming the United Nation’s commitment to achieve international co-operation in solving international problems of an economic, social, cultural, or humanitarian character,

Urging the cooperation of all States in the prevention of catastrophes caused by artificial intelligence,

Desiring to further facilitate the monitoring of advanced hardware, the avoidance of an artificial intelligence arms race, and the elimination and prevention of efforts to prematurely develop human-level artificial intelligence, artificial superintelligence, and other forms of highly dangerous artificial intelligence,

Have agreed as follows:

ARTICLE I

Definitions

For the purposes of this Treaty:

1. “Advanced hardware” means powerful computing semiconductor chips or integrated circuits that can be used to build artificial intelligence systems above the Danger Threshold.
2. “Algorithmic improvement” means advancements in artificial intelligence algorithms, methodologies, architectures, or techniques that lead to either: (a) a reduction in the computational resources, data, time, or cost required to develop advanced artificial intelligence systems or (b) an improvement in artificial intelligence capabilities.
3. “Artificial intelligence” means the following, together or separately:
   
   1. Any artificial system that performs tasks under varying and unpredictable circumstances without significant human oversight, or that can learn from experience and improve performance when exposed to data sets.
   2. An artificial system developed in computer software, physical hardware, or other context that solves tasks requiring human-like perception, cognition, planning, learning, communication, or physical action.
   3. An artificial system designed to think or act like a human, including cognitive architectures and neural networks.
   4. A set of techniques, including machine learning, that is designed to approximate a cognitive task.
   5. An artificial system designed to act rationally, including an intelligent software agent or embodied robot that achieves goals using perception, planning, reasoning, learning, communicating, decision-making, and acting.
   6. A machine-based system that is capable of influencing the environment by producing an output (predictions, recommendations or decisions) for a given set of objectives. It uses machine and/or human-based data and inputs to (i) perceive real and/or virtual environments; (ii) abstract these perceptions into models through analysis in an automated manner (e.g., with machine learning), or manually; and (iii) use model inference to formulate options for outcomes.
4. “Artificial general intelligence” or “human-level artificial intelligence” means artificial intelligence that achieves human-level performance at a wide variety of intellectual tasks, without being constrained to a narrow or specific domain of expertise.
5. “Artificial superintelligence” means artificial intelligence that exceeds human-level performance in most or all domains, potentially including general problem-solving, social skills, planning and strategic thinking, scientific research, and artificial intelligence development.
6. “Compute” means the processing power and other electronic resources used to train, validate, deploy, and run artificial intelligence algorithms and models.
7. “Dangerous artificial intelligence systems” includes collectively “human-level artificial intelligence”, “artificial general intelligence”, “artificial superintelligence”, and any artificial intelligence systems that pose severe global or national security risks.
8. “Floating-Point Operations” (FLOP) means single-precision (32-bit) floating point operations.
9. “Whistleblower” means any individual who reports unlawful or dangerous artificial intelligence development practices to the State within whose borders the unlawful or dangerous artificial development practices are carried out or to a trusted international agency

ARTICLE II

General Obligations

1. Each State Party undertakes to prohibit the civilian or military development, deployment, transfer, possession and use of artificial intelligence systems above the Moratorium Threshold.
2. Each State Party undertakes to regulate the development and use of artificial intelligence systems above the Danger Threshold but below the Moratorium Threshold such that any entity developing or using systems above the Danger Threshold (but below the Moratorium Threshold) must show that they are following appropriate regulations and safeguards. Examples include information security requirements, probabilistic risk assessments, predictions of dangerous capabilities, third-party auditing, and other regulations protecting safety and fundamental rights.
3. The Moratorium Threshold and the Danger Threshold shall initially be set based on the compute required to develop artificial intelligence systems (which serve as a proxy for the model’s capabilities). The Moratorium Threshold will start at 10ˆ24 Floating-Point Operations and the Danger Threshold will start at 10ˆ21 FLOP.
4. Each State Party undertakes to prohibit the development and use, for civilian or military purposes, of human-level artificial intelligence, artificial general intelligence (AGI), artificial superintelligence (ASI), or other forms of highly dangerous artificial intelligence systems.
5. Each State Party undertakes to implement comprehensive regulations to monitor artificial intelligence development, to report any instances in which an individual or group is suspected of developing or using one or many highly dangerous artificial systems or is suspected to be attempting to develop or use one or many highly dangerous artificial intelligence systems to the United Nations Security Council, and to investigate credible threats.

ARTICLE III

Threshold Revisions

1. One year after the entry into force of this Treaty, a conference of Parties shall be held in Geneva, Switzerland, in order to review the Moratorium Threshold.
2. At an interval at least once per year the State Parties will meet in Geneva, Switzerland, in order to review the Moratorium Threshold. The definitions of the Moratorium Threshold and Danger Threshold are recognized to be imperfect proxies that will be updated over time. It is further recognized that the development, acquisition, possession, or use of dangerous artificial intelligence systems will require fewer computing resources over time, due to algorithmic progress and other improvements (e.g., the discovery of new prompting techniques) to artificial intelligence development.

ARTICLE IV

Emergency response plans

1. Each State Party commits to developing and testing one or more emergency response plans in which States demonstrate the capacity to swiftly detect and halt dangerous artificial intelligence development (e.g., stop a training run before or immediately after it crosses the Moratorium Threshold) or stop the proliferation of a dangerous artificial intelligence model (e.g., withdraw application programming interface (API) access).
2. Each State Party undertakes to conduct tests of its emergency response plan or plans at regular intervals to ensure that States have the capacity to respond effectively in the event of an emergency.
3. Each State Party agrees to share information in good faith relating to the monitoring of artificial intelligence capabilities and development.

ARTICLE V

Monitoring and enforcement

1. Each State Party undertakes to take appropriate measures to ensure the enforcement of this treaty, including the development of the infrastructure required to enforce the Treaty.
2. Each State Party undertakes to self-report the amount and locations of large concentrations of advanced hardware to relevant international authorities.
3. Each State Party recognizes that the self-reporting procedure must allow for the comprehensive verification of advanced hardware in declared facilities in order to monitor that it is not being used to develop artificial intelligence above the Moratorium Threshold and to allow for the detection of any undeclared or secret facilities with large concentrations of advanced hardware.
4. Each State Party recognizes the need for the establishment of a protocol allowing for investigation by independent evaluators within their borders and undertakes to negotiate in good faith to that effect.

ARTICLE VI

Negotiations for creating an international organization for monitoring, enforcement, and research

1. Each State Party undertakes to engage in good-faith negotiations to create an international agency for the purpose of verification of the fulfillment of its obligations assumed under this Treaty. The primary purpose of this international agency would be to ensure that the provisions in the Treaty are effectively enforced.
2. The agency would also be responsible for researching highly powerful artificial intelligence systems, with the ultimate goal of understanding how to control highly powerful artificial in- intelligence systems and ensure that they are only ever developed for the benefit of the whole of humanity.
3. The agency would be responsible for adjusting the Moratorium Threshold and the Danger Threshold. The Moratorium Threshold may be lifted if the agency acquires compelling evidence that they are able to safely build and deploy human-level artificial intelligence and artificial superintelligence.

ARTICLE VII

Sharing the benefits from safe artificial intelligence

Each State Party undertakes to collaborate in good-faith for the establishment of effective measures to ensure that potential benefits from safe and beneficial artificial intelligence systems are distributed globally.

ARTICLE VIII

Communicating dangers and establishing whistleblower protections

1. Each State Party undertakes to establish and participate in an international hotline, allowing for direct communication between leaders in each State pertaining to matters of global security threats related to artificial intelligence.
2. Each State Party agrees to report evidence of dangerous artificial intelligence development, insights about dangerous capabilities, suspected noncompliance with the Treaty, and any other issue posing a threat to global security. 3. Each State Party undertakes to establish a similar communication channel for civilian artificial intelligence developers. Civilian artificial intelligence developers will be required to share evidence of dangerous artificial intelligence development, insights about dangerous capabilities, suspected noncompliance with the Treaty, and any information on any issue posing a potential threat to global security.
3. Each State Party undertakes to establish appropriate protections for whistleblowers who report unlawful or dangerous artificial intelligence development practices to the State or a trusted international entity.

ARTICLE IX

Prevention of an artificial intelligence arms race

Each State Party undertakes to pursue in good faith negotiations on effective measures relating to the cessation of an artificial intelligence arms race and the prevention of any future artificial intelligence arms race.

ARTICLE X

Review conferences

1. At least once per year after the entry into force of this Treaty, a conference of the State Parties shall be held in Geneva, Switzerland.
2. The purpose of the conference will be to review the operation of this Treaty to assure that the purposes of the Preamble and the provisions to the Treaty are being realized, to discuss possible changes to the Moratorium Threshold and Danger Threshold to account for artificial intelligence progress (described in Article II), and discuss other matters pertaining to global security threats from artificial intelligence.

ARTICLE XI

National regulations beyond the scope of the treaty

1. Nothing in this Treaty affects the right of any State or group of States to implement regulations based on definitions that include criteria other than a FLIP threshold (such as benchmark performance, parameter count, the domains in which the artificial intelligence can be applied, or the presence of certain dangerous capabilities).
2. Nothing in this Treaty affects the right of any State or group of States to implement regulations on artificial intelligence systems below the Moratorium and Danger Thresholds, or additional regulations applied to systems above the Danger Threshold (but below the Moratorium Threshold) that do not interfere with the obligations provided for by the Treaty.
3. Each State Party accepts the responsibility to impose its own regulations on artificial intelligence developers who are engaging, under said State Party’s jurisdiction, in activities that could pose national or global security threats.

ARTICLE XII

Settlement of disputes

When a dispute arises between two or more States Parties relating to the interpretation or application of this Treaty, the parties concerned shall consult together with a view to the settlement of the dispute by negotiation or by other peaceful means of the parties’ choice in accordance with Article 33 of the Charter of the United Nations.

ARTICLE XIII

Signature, ratification, entry into force, and withdrawal

1. This Treaty shall be open for signature to all States for signature before its entry into force.
2. This Treaty shall be subject to ratification, acceptance or approval by signatory States. The Treaty shall be open for accession.
3. This Treaty shall enter into force 60 days after the date of the deposit of the second instrument of ratification.
4. For States whose instruments of ratification, acceptance, approval or accession are deposited subsequent to the entry into force of this Treaty, it shall enter into force on the 30th day following the date of deposit of their instrument of ratification, acceptance, approval or accession.
5. Each State Party shall, in exercising its national sovereignty, have the right to withdraw from this Treaty if it decides that extraordinary events related to the subject matter of the Treaty have jeopardized the supreme interests of its country. It shall give notice of such withdrawal to the Depositary. Such notice shall include a statement of the extraordinary events that it regards as having jeopardized its supreme interests. Such withdrawal will only take effect 6 months after the date of receipt of the notification of withdrawal.
6. The Secretary-General of the United Nations is hereby designated as the Depositary of this Treaty.