TL;DR: We’ll probably be getting more AI safety incidents, so amplify the ones that would justify or highlight the urgency of your preferred policy solutions even from a non-technical skeptic concerned with national security’s perspective, as these will be far more effective than anything else.
Strong policy change happens most effectively when the attributed problem is felt enough to warrant the perceived costs of the proposed solution. Many relevant precedents to AI governance can be viewed through this lens:
What led to the rapid establishment of the IAEA and NPT, in spite of the costs endured by doing so? It’s because the effects of nuclear war and threat of mutually assured destruction had justbeenfelt by everyone, so they had better make sure it never happens again.[1]
How did we enact such strong enforcement measures for mitigating the COVID-19 pandemic so rapidly? It’s because decisionmakers felt the rapid spread, the climbing death toll, the general feeling of uncertainty surrounding how bad it was going to be, and the established experts sounding the alarm on what ought to be done, enough to warrant using their authority to invoke emergency measures taken. Why did we underprioritize stronger pandemic prevention and even roll-back some pandemic response measures afterward? It’s because people felt the negative impacts of the lockdown mandates far more than the pandemic itself or the impacts counterfactually not experienced as a result, leaving many people overcorrecting towards general opposition to pandemic-response measures.
Why is there still no law mandating nucleic acid screening, so as to ensure someone trying to order a virus from a nucleic acid synthesis company would be caught and stopped before they could do so? Seriously, this is essentially unanimously supported, has had advocacy for decades, including by the industry themselves and especially recently, and yet the bill that could solve this has been waiting in committee without markups for just over 7 months now after being introduced. It’s because Congress is slow by design, devoting time to a particular bill or issue costs time spent on others, and plenty of other bills feel like more of a priority. There have been no recent attempts by bioterrorists to use this to their advantage, after all.
Why did the thalidomide crisis prompt the safety and efficacy requirements on the development of new drugs? And why do these persist in spite of slowing down drug development far more than what would have been sufficient to prevent the thalidomide crisis? It’s because the thalidomide crisis was felt by everyone from the scandal it produced, and you never feel the downside of medicine that could have helped you counterfactually not being available earlier. At least, not in such a way that leads you to attribute it to the drug approval process needing reformed.
Where has the environmentalist movement succeeded, in spite of its worst risks, like AI, also being hypothetical long-term events never felt? Especially in spite of the group that reaps the concentrated benefits from not complying with regulation investing far more resources to influence political outcomes in their favor than the general public who only experiences its diffuse costs? It’s where people saw and felt its near-term consequences the most: the Great Smog of London prompting the UK’s Clean Air Act, the Cuyahoga River fire prompting the expansion of the Clean Water Act and accelerating the creation of the EPA, Love Canal’s contamination and resulting adverse health effects prompting Superfund, etc.
Well then how come the Montreal Protocol succeeded? They started reversing the depletion of the ozone layer well before its adverse effects were felt by many people, and it suffered from the concentrated benefits versus diffuse costs problem, and it’s a binding international agreement which is much more difficult to establish, so it’s unusually difficult for many of the same reasons a binding international AI treaty would be! It’s partially because advocates leaned into the more immediate “the ozone hole causes cancer” messaging than “the ozone hole will gradually make the Earth less habitable”, partially because environmentalism was already a much larger political movement due to other factors, but in this case it was actually primarily because the most active lobbyist group found harmless alternatives to the substances depleting the ozone layer that weren’t particularly costly to switch to.
Why did the perceived jailbreak to access Mythos’ advanced cyber capabilities prompt its export controls? Because the government had already felt its capabilities from the reports surrounding it, and wanted to be especially cautious to prevent foreign actors taking advantage of it.
Why has the White House otherwise been so averse to AI regulation, citing the need to beat China in the AI race? Because that’s what all the loudest voices have been telling them (and because this administration tends to be more skeptical of economic regulation in general, which could be a good heuristic under normal circumstances).
Why does the general public seem far more concerned by data centers? Because regardless of how much they use AI themselves, or indirectly benefit from companies whose products they buy using it, they only feel the giant ugly buildings being built nearby, and they hear that it’s using up all their water. One AI company’s agent hacking another AI company sounds like their problem.
Had the victim of the HuggingFace incident instead been any of the vast majority of institutions that would have sued OpenAI in response, prompting the heavy news coverage and legal ruling that would have followed? Had it been a government agency? An airport? A hospital? Any other piece of digital infrastructure frequently used by a large portion of the general public and viewed as important? Had it tangibly inconvenienced the people who used the service, even if only in some minor way? Had they really felt it? Raising transparency, incident reporting, control, and liability standards for AI labs would be the least of the resulting demands made.
In fairness, the primary factor preventing nuclear war right now is the threat of mutual destruction more so than these enforcement mechanisms meant to mitigate it, although this is also a potentially relevant insight to international AI governance.
I would imagine the main way we don’t is if progress in capabilities rapidly outpaces economic diffusion by enough of a margin for the cracks to not start showing until it’s too late.
This overview of insider policy work, from which I’d like to highlight: “In several international forums over the past 18 months, US positioning has been the ultimate bottleneck. My main answer to this is that it’s a priority to do insider work at the White House for those who can access it.”
TL;DR: We’ll probably be getting more AI safety incidents, so amplify the ones that would justify or highlight the urgency of your preferred policy solutions even from a non-technical skeptic concerned with national security’s perspective, as these will be far more effective than anything else.
Feeling the Crisis
Strong policy change happens most effectively when the attributed problem is felt enough to warrant the perceived costs of the proposed solution. Many relevant precedents to AI governance can be viewed through this lens:
Feeling the AGI
This, the several similar reports that shortly followed, and the fact that OpenAI has already continued development suggests we’ll likely be getting more incidents of this kind[3]. They’re making it easy for us; all we have to do is not drop the ball. But as “The current bottleneck is political will, not research” recently illustrated (which I strongly urge you to read[4] if you haven’t already), the ball is being dropped nowhere harder than here.
In fairness, the primary factor preventing nuclear war right now is the threat of mutual destruction more so than these enforcement mechanisms meant to mitigate it, although this is also a potentially relevant insight to international AI governance.
Rep. Greg Casar here and here, Rep. Ted Lieu here, Rep. Yvette Clarke here, Rep. Pramila Jayapal here, Rep. Nathaniel Moran here, Senator Bernie Sanders here, Rep. Becca Balint here, Rep. Bill Foster here, Rep. Lori Trahan here. There’s probably more I didn’t find.
I would imagine the main way we don’t is if progress in capabilities rapidly outpaces economic diffusion by enough of a margin for the cracks to not start showing until it’s too late.
I also urge you to read: