The following are some very short stories about some of the ways that I expect AI regulation to make x-risk worse.

I don't think that each of these stories will happen. Some of them are mutually exclusive, by design. But, conditional upon AI heavy regulations passing, I'd strongly expect the dynamics pointed at in more than one of them to happen.

Pointing out potential costs of different regulations is a necessary part of actual deliberation about regulation; like all policy debates, AI policy debates should not appear one-sided. If someone proposes an AI policy, and doesn't actively try to consider potential downsides -- or treats people who bring up these downsides as an opponent -- they are probably doing something closer to propaganda than actual policy work.

All these stories have excessive detail; these are archetypes meant to illustrate patterns more than they are predictions. Before mentioning one fix that would stop a problem, consider whether it would be subject to similar problems in the same class.

(a) Misdirected Regulations Reduce Effective Safety Effort; Regulations Will Almost Certainly Be Misdirected

"Alice, have you gotten the new AI model past the AMSA [AI and ML Safety Administration] yet?" said Alex, Alice's manager. "We need their approval of the pre-deployment risk-assessment to launch."

"I'm still in correspondence with them," said Alice, wincing. "The model is a little more faceblind than v4 -- particularly in the case of people with partially shaved heads, facial tattoos, or heavy piercings -- so that runs into the equity clauses and so on. They won't let us deploy without fixing this."

"Sheesh," said Alex. "Do we know why the model is like this?"

Alice shrugged.

"Well, getting this model past the AMSA is your number one priority," Alex said. "Check if you can do some model surgery to patch in the facial recognition from v4. It should still be compatible."

"Alex," said Alice. "I'm not really worried about the face-blindness. It's a regression, but it should have minimal impact."

"I mean, I agree," grinned Alex, "but don't put that in an email."

"But," Alice continued, hesitantly, "I'm really concerned about some of the capabilities of the model, which I don't think we've explored completely. I know that it passed the standard safety checks, and I know I've explored some of its capabilities, but I think this is much much more important than -- "

"Alice," interrupted Alex. "We've gone over this before. You're on the Alignment and Safety Team. That means part of your job is to make sure the model passes AMSA standards. That's what the Alignment and Safety team does."

"But the AMSA regulations went through Congress before Ilya's Abductor was even invented!" Alice said. "I know the regs are out of date; you know the regs out of date; everyone knows they're out of date, but they haven't even begun to change yet!--"

"Alice," interrupted Alex again. "Sorry for interrupting. And look, I'm really sorry about the situation in general. But you've spent two whole months on this already. I let you bring in Aaron on the problem, and he spent a whole month. Neither of you have found anything dangerous."

He continued: "I've shielded you from pressure from my boss for a while now. He was understanding the first three weeks, impatient the next three weeks, and has been increasingly irritated since. I wish we had more time for you, but we can't just ignore the AMSA -- it's the law. I need you to work on this and only this, or you're fired."

I expect actual instances of misdirected safety effort from safety laws to be far more universal, and only moderately more hidden, than is indicated in this dialogue.

If you think this is unlikely, consider IRBs, and consider that essentially there are nearly as many theories about what AI safety should look like as there are AI safety theorists.

(b) Regulations Generally Favor The Legible-To-The-State

"Our request for compute has been denied by the AMSA. Again." said Barbara. "No actual explanation for why, of course."

"Fuck," said Brian. "Fuck. Why!? We want to do a zero-risk interpretability experiment. Why on earth are OpenMind and DeepAI -- and Lockheed Martin, and Palantir, and Anduril -- getting their requests for huge GPU runs approved, and not us?"

Barbara looked at him blankly.

"Is that a rhetorical question?" she said.

"No!"

"I mean... probably because the regulators see those entities as national assets, and not us. Probably because the regulations were specifically written to fit the kind of requests for compute that such organizations would tend to make. Like they just fill out a form on the web with most of the fields pre-populated. And probably because these organizations have long-standing relationships with the government," she said.

"But the bureaucracy was brought into existence to help fight x-risk, which means funding interpretability!" said Brian. "But their access to compute was scarcely curtailed!"

Barbara shrugged, her mouth quirking.

"You wouldn't make that kind of mistake about an AI, you know," she said.

"The mistake -- oh, thinking that it will act for a specific goal because it was made to accomplish that goal."

Brian deflated.

Barbara spoke: "The AMSA bureaucracy came into existence because of a wide-spread political coalition with mutually incompatible goals. Many of the coalition's goals were contradictory, or impossible, or otherwise dumb, which is typical for politics. You shouldn't expect that kind of an entity to have coherent ends that it acts to execute. You shouldn't expect it to even have goals; it has procedures. Which we don't fit into particularly well, and with which we are now stuck."

"But we should be able to change it now," Brian said, without conviction.

"Government's aren't exactly corrigible either," Barabara sighed. "I'll help you with the fourth draft."

Consider that the National Environmental Protection Act as enforced has exemptions to speed oil and gas projects but not for many similar renewable energy projects.

(c) Heavy Regulations Can Simply Disempower the Regulator

Following Christopher Nolan's surprising smash hit, Diamondoid Bacteria -- which ends with a montage of everyone on Earth falling over dead in the same five seconds, after an AI has infected them with a nanotechnological virus -- the US passes unprecedented national anti-AI regulation.

The atmosphere of fear demands intense action; so the regulation is deeply conservative and in every case of ambiguity bans things rather than permits them. Anthropic, OpenAI, and other AI labs are all simply shut down. Interconnect between more than a handful of H100s is banned. TSMC and Nvidia stock plunges.

Nevertheless, the US cannot rule out AI research everywhere by simple fiat. It isn't the 90s anymore; it's a multi-polar world.

India, China, and a handful of other countries continue to do work with AI -- except the researchers there are far more cautious about how they represent themselves publicly. There's a small diaspora of AI researchers from the US to India and Australia, countries that the US cannot pressure because it needs them on their side in geopolitical conflict with China. And in fact the US starts to decline -- even more quickly -- relative to the influence of China, India, and others because it has effectively banned the most transformative technology of the 2020s. New technologies emerge at an increasing rate from China, India, and other countries, increasing their geopolitical power, and accelerating the prexisting relative power decline of the US.

In 1 to 4 years, we're in the same situation we were before, except AI research is more spread out and now conducted in the countries that are most hostile to alignment and least concerned for safety. Nearly all of the safety work -- conducted on low-power systems in the US in the interim -- turns out to be useless, given how far pre-AGI the systems were, so it's a strictly worse situation.

I don't think exact world is not particularly likely; but I do think this or something like this could happen, in a world where fear drives policy making and where the most conservative / worried AI policy people have their way.

(d) Regulations Are Likely To Maximize The Power of Companies Pushing Forward Capabilities the Most

"So. I'm unemployed too now," Darren said, collapsing onto the couch on the porch. He stared at the moon rising over the city.

"Another one bites the dust," said David, puffing cigarette smoke into the night.

He didn't one ask why Darren was unemployed. He knew the answer already. In the last year, the three largest corporations in the US had grown at a cancerous rate, swallowing the economic spaces of hundreds of other companies. All of them had been AI companies; you could describe them now more as "everything companies." They now produced more than 50% of literally all things manufactured in the US by value; in a handful of months it would be more than 60%.

Although, by now, "corporation" somewhat of a misnomer -- they handed out food allotments, supplied entertainment, enforced the peace, often provided housing and -- so far as Darren could tell -- dictated policy to the increasingly moribund government of the United States.

If you didn't have an artificial intelligence, you couldn't keep up.

"Once I did -- 'open source AI work'," Darren said, sadly, with air quotes to distance himself from the phrase. He didn't mean it as a claim about previous success; it was an expression of sadness for a lost past.

"Doubt you ever had a chance against the boys with with the billions," said David.

"Fellas with the FLOPs," said Darren.

"Guys with the GPUs," said David. "Don't look at me like that, I know that one doesn't work."

"But look," Darren said irritably, "they didn't always rule the world. There were open source machine-learning models that were useful once."

He continued:

"You could download a general-purpose model, shove it into a robot dog, and it would work in less than an hour. You could run a code-completion model on your very own GPUs. People were doing open-source interpretability work to steer these things, because they could see the weights; it helped you modify them however you wanted. Everyone was building small models that could compete with big ones on very specific domains, if not general ones. And were hundreds of specific domains that people were training them on."

He drank some more vodka.

"The open source ones tended to be small models. They were far less general than the models of the big guys. But it was still all shut down as too hazardous in `25."

He continued: "They got a law passed, stating that models trained on too many flops were forbidden without a license. Which meant all the useful open-source models were forbidden; anything powerful had to be behind an API, to meet all the requirements. Which meant anything useful had to be behind an API."

He sighed.

"Then everyone had to purchase intelligence from the big guys because they were the only game in town. Their profits fucking doubled. Their return on investment doubled. Why wouldn't it double, we rewarded those fuckers with a monopoly because they promised they'd make it all safe, even though they had said they were trying to make God, and open source was just usually trying to make... some part of a man. AI was already rewarding capital because it was a machine that took capital and printed out labor, and we poured gasoline on that fire."

Darren sighed again.

"Well," said David sarcastically, "you forget the the advantages of maximizing the intelligence differential between men; only a few companies having power made co-ordination easier."

"Christ," said Darren. "Yeah coordination is always easier if only a few people have power. Historically undeniable."

"Definitely aren't any other disadvantages."

"Nope."

In the distance, beyond the decaying human-occupied part of the city, they could see the wall of a factory being lifted by robot-controlled drones. The robots worked 24/7, of course. A time-lapse from here would have shown whole new factories rising in weeks.

"Wouldn't matter if you had a whole farm of GPUs now," said David.

"No, it would not," said Darren. "But once upon a time it, might have."

Open source models have some disadvantages. Policies trying to shut down open source models also have some disadvantages. Thinking about these disadvantages is important for deciding if shutting down open source is an amazing idea, or a really shitty idea. I have tried to gesture at a small handful of these disadvantages here; I think that there are many more.