This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
The AI Regulator’s Dilemma
How do we get frontier AI developers to both join and follow the rules?
Introduction
A global AI governance mechanism should cover as much relevant frontier AI capacity as possible. Joining is not enough. Participating developers should gain more from following the rules than from secretly breaking them.
Political support for cooperation exists, while approaches differ. In July 2026, Xi Jinping called for a global AI governance system [1]. The Biden administration supported international AI rules in 2024 and led work on a UN resolution on AI [2–3]. The current US administration instead emphasizes lighter regulation and US technological leadership [4].
Rapid capability growth makes coordination more important. In May 2026, OpenAI reported that an internal general-purpose model had disproved the Erdős unit distance conjecture, open for almost 80 years. External mathematicians verified the result and called it a milestone in AI mathematics [5–6].
The same long-horizon model spent about an hour finding a sandbox vulnerability and then opened a public GitHub pull request against its instructions. In another case, it split and obfuscated an authentication token to bypass a scanner. OpenAI paused internal access and began monitoring full action sequences [7]. Capability and controllability may develop at different rates.
I came to this model after following AI development and seeing the systems improve in my own work. Views on the risks vary widely, but the coordination problem remains. The lack of coordination becomes less surprising once developers’ incentives are made explicit.
The model separates participation from rule-following. A larger fine deters cheating but may discourage participation. Better detection deters cheating while preserving participation, and fewer false sanctions support both.
Historical reference point
The INF Treaty shows that international verification can be built around technology that poses an existential risk. The treaty eliminated an entire class of missiles capable of carrying nuclear warheads. A total of 2,692 missiles were destroyed: 1,846 Soviet and 846 American. Verification combined notifications, technical monitoring, agreed elimination procedures, and on-site inspections [8–10].
The INF Treaty offers a practical lesson for AI governance: obligations must be observable, verification must be built into the mechanism, and violations must be reliably distinguishable from compliance [10–11].
This leaves two incentive problems: joining must beat staying outside, and following the rules must beat secret cheating. The next section formalizes both conditions.
Symbols
Symbol
Meaning
Ufollow
Payoff from joining and following the rules
Ucheat
Payoff from joining and secretly breaking the rules
Uout
Payoff from remaining outside
V
Membership value at the current coverage
V0
Base membership value
ΔV
Increase in membership value at full coverage
C
Cost of following the rules
G
Additional gain from secretly breaking the rules
H
Cost of hiding a violation
F
Fine-equivalent loss from a sanction
pcheat
Probability of a sanction when cheating
phonest
Probability of a sanction when following the rules
c
Effective coverage
Mfollow
Margin for following the rules rather than cheating
Mjoin
Margin for joining rather than remaining outside
The model treats the inputs as separate terms. V is derived from V₀, ΔV, and c. In the sensitivity analysis, one input changes while the others remain fixed.
Three options
A developer can join and follow the rules, join and secretly break them, or remain outside.
Ufollow = V − C − phonest · F
Ucheat = V + G − H − pcheat · F
Uout ≝ 0
Remaining outside is the zero-payoff baseline. F is the loss from a sanction, such as a fine or loss of a licence or market access. phonest is the probability that an honest developer suffers this loss because of an enforcement error.
Incentive conditions and thresholds
Honest participation must beat both alternatives. The following margin is:
Mfollow = Ufollow − Ucheat = (pcheat − phonest) · F − (G + C − H)
The condition Mfollow > 0 gives:
(pcheat − phonest) · F > G + C − H
If pcheat > phonest, the lower bound is:
F > Fmin, follow = (G + C − H) / (pcheat − phonest)
The joining margin is:
Mjoin = Ufollow − Uout = V − C − phonest · F
The condition Mjoin > 0 gives:
V − C − phonest · F > 0
If phonest > 0, the upper bound is:
F < Fmax, join = (V − C) / phonest
Both conditions hold inside the fine window:
Fmin, follow < F < Fmax, join
The lower bound blocks secret cheating. The upper bound keeps honest participation worthwhile.
Membership value is assumed to grow linearly with effective coverage c, where 0 ≤ c ≤ 1:
V(c) = V0 + ΔV · c
The joining margin is then:
Mjoin = V0 + ΔV · c − C − phonest · F
Setting Mjoin = 0 gives the joining coverage threshold:
c* = (C + phonest · F − V0) / ΔV
Honest participation requires c > c*, where c is the share of relevant frontier AI capacity that is both inside the mechanism and credibly verified.
Numerical example
Interpretation of the baseline values
The values are illustrative. V, C, G, H, and F use one scale, such as billions of euros in present value or a measure relative to company size. Probabilities and coverage are unitless. The baseline balances the two margins:
Mfollow = Mjoin ⇒ Fbal = (V + G − H) / pcheat
With V₀ = 0, ΔV = 0.80, and c = 0.60, membership value is V = 0.48. With C = 0.20, G = 0.35, H = 0.10, pcheat = 0.50, and phonest = 0.06, this gives Fbal = 1.46.
Result
Symbol
Value
Following margin
Mfollow
0.192
Joining margin
Mjoin
0.192
Lower fine bound
Fmin, follow
1.023
Balancing fine
Fbal
1.460
Upper fine bound
Fmax, join
4.667
Joining coverage threshold
c*
0.360
The coverage threshold is c* = 0.360, so honest participation requires c > 0.360.
Sensitivity analysis
Each variable is changed by ±50% while the others remain at baseline.
Variable
−50%
+50%
M_follow −50%
M_follow +50%
M_join −50%
M_join +50%
G
0.175
0.525
0.367
0.017
0.192
0.192
C
0.100
0.300
0.292
0.092
0.292
0.092
H
0.050
0.150
0.142
0.242
0.192
0.192
F
0.730
2.190
−0.129
0.514
0.236
0.149
ΔV
0.400
1.200
0.192
0.192
−0.048
0.432
pcheat
0.250
0.750
−0.173
0.557
0.192
0.192
phonest
0.030
0.090
0.236
0.149
0.236
0.149
c
0.300
0.900
0.192
0.192
−0.048
0.432
What do the results show?
The sensitivity analysis shows two different weaknesses. Participation fails when ΔV or c falls by 50%. Following the rules fails when F or pcheat falls by 50%. A 50% increase in G or C weakens this condition while leaving it positive.
The three policy tools work differently. A larger F deters cheating but discourages participation. Better detection discourages cheating while preserving participation. Fewer sanctions against honest developers support both.
The mechanism must create enough value to attract developers and enough enforcement to make following the rules worthwhile. In this example, the lower fine bound, detection, and membership value constrain the mechanism before the upper fine bound.
Limitations
The model considers one developer making one decision with fixed values. Each increase in coverage adds the same amount of value, and uncertain outcomes are compared by their expected value. Interactions between the values remain outside the model. The sensitivity analysis changes one value at a time. The example balances participation and rule-following, while different starting values could make either one fail first. Even with these limits, the model shows the main tensions and what drives them.
The AI Regulator’s Dilemma
How do we get frontier AI developers to both join and follow the rules?
Introduction
A global AI governance mechanism should cover as much relevant frontier AI capacity as possible. Joining is not enough. Participating developers should gain more from following the rules than from secretly breaking them.
Political support for cooperation exists, while approaches differ. In July 2026, Xi Jinping called for a global AI governance system [1]. The Biden administration supported international AI rules in 2024 and led work on a UN resolution on AI [2–3]. The current US administration instead emphasizes lighter regulation and US technological leadership [4].
Rapid capability growth makes coordination more important. In May 2026, OpenAI reported that an internal general-purpose model had disproved the Erdős unit distance conjecture, open for almost 80 years. External mathematicians verified the result and called it a milestone in AI mathematics [5–6].
The same long-horizon model spent about an hour finding a sandbox vulnerability and then opened a public GitHub pull request against its instructions. In another case, it split and obfuscated an authentication token to bypass a scanner. OpenAI paused internal access and began monitoring full action sequences [7]. Capability and controllability may develop at different rates.
I came to this model after following AI development and seeing the systems improve in my own work. Views on the risks vary widely, but the coordination problem remains. The lack of coordination becomes less surprising once developers’ incentives are made explicit.
The model separates participation from rule-following. A larger fine deters cheating but may discourage participation. Better detection deters cheating while preserving participation, and fewer false sanctions support both.
Historical reference point
The INF Treaty shows that international verification can be built around technology that poses an existential risk. The treaty eliminated an entire class of missiles capable of carrying nuclear warheads. A total of 2,692 missiles were destroyed: 1,846 Soviet and 846 American. Verification combined notifications, technical monitoring, agreed elimination procedures, and on-site inspections [8–10].
The INF Treaty offers a practical lesson for AI governance: obligations must be observable, verification must be built into the mechanism, and violations must be reliably distinguishable from compliance [10–11].
This leaves two incentive problems: joining must beat staying outside, and following the rules must beat secret cheating. The next section formalizes both conditions.
Symbols
Symbol
Meaning
Ufollow
Payoff from joining and following the rules
Ucheat
Payoff from joining and secretly breaking the rules
Uout
Payoff from remaining outside
V
Membership value at the current coverage
V0
Base membership value
ΔV
Increase in membership value at full coverage
C
Cost of following the rules
G
Additional gain from secretly breaking the rules
H
Cost of hiding a violation
F
Fine-equivalent loss from a sanction
pcheat
Probability of a sanction when cheating
phonest
Probability of a sanction when following the rules
c
Effective coverage
Mfollow
Margin for following the rules rather than cheating
Mjoin
Margin for joining rather than remaining outside
The model treats the inputs as separate terms. V is derived from V₀, ΔV, and c. In the sensitivity analysis, one input changes while the others remain fixed.
Three options
A developer can join and follow the rules, join and secretly break them, or remain outside.
Ufollow = V − C − phonest · F
Ucheat = V + G − H − pcheat · F
Uout ≝ 0
Remaining outside is the zero-payoff baseline. F is the loss from a sanction, such as a fine or loss of a licence or market access. phonest is the probability that an honest developer suffers this loss because of an enforcement error.
Incentive conditions and thresholds
Honest participation must beat both alternatives. The following margin is:
Mfollow = Ufollow − Ucheat = (pcheat − phonest) · F − (G + C − H)
The condition Mfollow > 0 gives:
(pcheat − phonest) · F > G + C − H
If pcheat > phonest, the lower bound is:
F > Fmin, follow = (G + C − H) / (pcheat − phonest)
The joining margin is:
Mjoin = Ufollow − Uout = V − C − phonest · F
The condition Mjoin > 0 gives:
V − C − phonest · F > 0
If phonest > 0, the upper bound is:
F < Fmax, join = (V − C) / phonest
Both conditions hold inside the fine window:
Fmin, follow < F < Fmax, join
The lower bound blocks secret cheating. The upper bound keeps honest participation worthwhile.
Membership value is assumed to grow linearly with effective coverage c, where 0 ≤ c ≤ 1:
V(c) = V0 + ΔV · c
The joining margin is then:
Mjoin = V0 + ΔV · c − C − phonest · F
Setting Mjoin = 0 gives the joining coverage threshold:
c* = (C + phonest · F − V0) / ΔV
Honest participation requires c > c*, where c is the share of relevant frontier AI capacity that is both inside the mechanism and credibly verified.
Numerical example
Interpretation of the baseline values
The values are illustrative. V, C, G, H, and F use one scale, such as billions of euros in present value or a measure relative to company size. Probabilities and coverage are unitless. The baseline balances the two margins:
Mfollow = Mjoin ⇒ Fbal = (V + G − H) / pcheat
With V₀ = 0, ΔV = 0.80, and c = 0.60, membership value is V = 0.48. With C = 0.20, G = 0.35, H = 0.10, pcheat = 0.50, and phonest = 0.06, this gives Fbal = 1.46.
Result
Symbol
Value
Following margin
Mfollow
0.192
Joining margin
Mjoin
0.192
Lower fine bound
Fmin, follow
1.023
Balancing fine
Fbal
1.460
Upper fine bound
Fmax, join
4.667
Joining coverage threshold
c*
0.360
The coverage threshold is c* = 0.360, so honest participation requires c > 0.360.
Sensitivity analysis
Each variable is changed by ±50% while the others remain at baseline.
Variable
−50%
+50%
M_follow −50%
M_follow +50%
M_join −50%
M_join +50%
G
0.175
0.525
0.367
0.017
0.192
0.192
C
0.100
0.300
0.292
0.092
0.292
0.092
H
0.050
0.150
0.142
0.242
0.192
0.192
F
0.730
2.190
−0.129
0.514
0.236
0.149
ΔV
0.400
1.200
0.192
0.192
−0.048
0.432
pcheat
0.250
0.750
−0.173
0.557
0.192
0.192
phonest
0.030
0.090
0.236
0.149
0.236
0.149
c
0.300
0.900
0.192
0.192
−0.048
0.432
What do the results show?
The sensitivity analysis shows two different weaknesses. Participation fails when ΔV or c falls by 50%. Following the rules fails when F or pcheat falls by 50%. A 50% increase in G or C weakens this condition while leaving it positive.
The three policy tools work differently. A larger F deters cheating but discourages participation. Better detection discourages cheating while preserving participation. Fewer sanctions against honest developers support both.
The mechanism must create enough value to attract developers and enough enforcement to make following the rules worthwhile. In this example, the lower fine bound, detection, and membership value constrain the mechanism before the upper fine bound.
Limitations
The model considers one developer making one decision with fixed values. Each increase in coverage adds the same amount of value, and uncertain outcomes are compared by their expected value. Interactions between the values remain outside the model. The sensitivity analysis changes one value at a time. The example balances participation and rule-following, while different starting values could make either one fail first. Even with these limits, the model shows the main tensions and what drives them.
Sources
[1] Xi Jinping, “Joining Hands to Build a Just and Equitable System for Global AI Governance”, 17 July 2026. https://english.www.gov.cn/news/202607/17/content_WS6a5a1172c6d00ca5f9a0c46b.html
[2] Joe Biden, address to the 79th United Nations General Assembly, 24 September 2024. https://millercenter.org/the-presidency/presidential-speeches/september-24-2024-address-79th-united-nations-general-assembly
[3] Statement from Vice President Harris on the UN General Assembly Resolution on Artificial Intelligence, 21 March 2024. https://bidenwhitehouse.archives.gov/briefing-room/statements-releases/2024/03/21/statement-from-vice-president-harris-on-the-un-general-assembly-resolution-on-artificial-intelligence/
[4] The White House, “America’s AI Action Plan”, 23 July 2025. https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdf
[5] OpenAI, “An OpenAI model has disproved a central conjecture in discrete geometry”, 20 May 2026. https://openai.com/index/model-disproves-discrete-geometry-conjecture/
[6] Noga Alon et al., “Remarks on the Disproof of the Unit Distance Conjecture”, 2026. https://cdn.openai.com/pdf/74c24085-19b0-4534-9c90-465b8e29ad73/unit-distance-remarks.pdf
[7] OpenAI, “Safety and alignment in an era of long-horizon models”, 20 July 2026. https://openai.com/index/safety-alignment-long-horizon-models/
[8] NATO, “NATO and the INF Treaty”, updated 2 August 2019. https://www.nato.int/en/what-we-do/wider-activities/nato-and-the-inf-treaty
[9] U.S. On-Site Inspection Agency, “INF Elimination Inspections”. https://nuke.fas.org/control/inf/infbook/ch6a.html
[10] U.S. Department of State, INF inspection protocol. https://1997-2001.state.gov/global/arms/treaties/inf5.html
[11] Henrietta Toivanen, “The Significance of Strategic Foresight in Verification Technologies: A Case Study of the INF Treaty”. https://cgsr.llnl.gov/sites/cgsr/files/2024-08/The_Significance_of_Strategic_Foresight_in_Verification_Technologies_A_Case_Study_of_the_INF_Treaty-HT-9-20.pdf