I moderately agree here but I still think the primary factor is centralization of the value chain. The more of the value chain is centralized, the easier it is to control. My guess we can make this argument more formalized by thinking of things in terms of a dependency graph -- if we imagine the economic process from sand + energy -> DL models then the important measure is the centrality of the hubs in this graph. If we can control and/or cut these hubs, then the entire DL ecosystem falls apart. Conveniently/unfortunately this is also where most of the economic profit is likely to be accumulating by standard industrial economic laws, and hence this is also where there will be the most resources resisting regulation.
TLDR: Slowing AI is easiest if AI companies are horizontally integrated, but not vertically integrated.
Some people have expressed a preference for there being only one leading AI company,[1] or for leading AI companies to become better at coordinating with one another.[2] If there is one company which has a clear lead and that company is safety conscious, then it could move cautiously while developing increasingly powerful AI systems. Having multiple leading AI companies could lead to a race, and these companies might be less cautious in order to get ahead of their competitors.
It is not clear whether the incentives actually make developing powerful AI a race.[3] Even if they are not, if people believe that there is a race, they might accelerate AI development. If there were fewer or no competitors, companies would be less likely to believe that there is an AI arms race.
This is an argument in favor of horizontal integration. It would be better for AI companies to merge or cooperate with their competitors.
It is not an argument in favor of vertical integration. It would be easier to slow the development of potentially dangerous AI if no one company controlled all or most of the supply chain for developing AI.
Consider a hypothetical horizontally integrated, but not vertically integrated, AI sector:[4] There is one company which produces GPUs. A different company writes and maintains operating systems. A third company collects training data. A fourth company trains base models. A fifth company does fine tuning.
This supply chain has five bottlenecks. Any one of them could decide that AI systems are getting scary and decide to slow things down. Since each level is horizontally integrated, actors at other levels cannot turn to their competitors to replace them. It does not seem like any one of the five companies could completely stop further development, because there are multiple ways to make progress. Some of them are better positioned to slow development than others, but any one of them could make further developments harder for other levels of the supply chain.
If the AI sector were completely integrated, it would remove the potential race dynamics, but it also makes everything depend on a single actor. If that actor is not cautious, then other private actors would have more limited ways to respond. In a not vertically integrated AI sector, only one of several companies needs to be safety cautious.
The more vertically integrated a sector is, the less individual companies depend on other companies to produce their products. Vertical integration makes it easier for a single company to act alone.
Current antitrust legislation typically assumes that vertical integration is acceptable and horizontal integration is problematic.[5] This is true whenever the technological progress is unambiguously good. We want competition to accelerate progress and lower prices. But if a technology is a potential x-risk, then accelerating progress could be bad. In this situation, horizontal integration is good and vertical integration is bad.
“Speculatively, DeepMind hoped to get all the AI talent in one place, led by safety-conscious people, so that they could double-check things at their leisure instead of everyone racing against each other to be first.”
Scott Alexander. Why Not Slow AI Progress? Astral Codex Ten. (2022) https://astralcodexten.substack.com/p/why-not-slow-ai-progress.
“EA and AI safety have invested a lot of resources into building our ability to get coordination and cooperation between big AI labs.”
Evan Hubinger. AI coordination needs clear wins. AI Alignment Forum. (2022) https://www.alignmentforum.org/posts/vavnqwYbc8jMu3dTY/ai-coordination-needs-clear-wins.
Katja Grace. AI Is Not an Arms Race. Time. (2023) https://time.com/6283609/artificial-intelligence-race-existential-threat/.
This is just one way of having the sector be not vertically integrated. There are other possible ways to split up the supply chain, some of which might be better than this.
“Challenges of vertical mergers are generally more difficult than horizontal merger challenges.”
Matthew Lane. Antitrust in 60 Seconds: Vertical vs. Horizontal Mergers. Disruptive Competition Project. (2018) https://www.project-disco.org/competition/111918-antitrust-60-seconds-vertical-vs-horizontal-mergers/.