Common to many models of AGI development is the view that AGI (and a possible artificial superintelligence) is developed by a single firm, with decision making power centralised in a small number of stakeholders. We argue here that this view is misleading even in scenarios where AI progress occurs rapidly and discontinuously via an intelligence explosion. This is for two key reasons

A leading AI lab would still be accountable to its investors and compute partners
- Lab CEOs may be very powerful agents but they are still bound to their principals
Monetisation of more powerful AI models will require more decentralised ecosystems and cannot be effectively accomplished within a single firm, reducing the concentration of economic power in a single lab
- Trickier pathways to monetisation will limit labs ability to increase their internal compute consumption and slow overall progress

The AI 2027 scenario is the best and most fleshed out case for an intelligence explosion dynamic, providing concrete forecasts and arguments that this piece will look to question. AI 2027 features few economic constraints, with the leading lab understood to have effectively unlimited access to capital and rapid revenue growth. I argue that economics will likely act as an important brake on the pace of AI progress. Developing an AGI will require an ecosystem of actors providing capital, supply chains and end-demand. This ecosystem will affect the decision making capabilities of the leading lab, likely pushing towards greater focus on monetisation and competition in the market, requiring more compute dedicated to external needs, reducing the potential for internal recursive development.

Labs don’t exist in isolation

The AI 2027 largely assumes that all the key decisions by the leading AI lab are undertaken by its executive team, with little external input in this process. Once models progress to significantly greater capabilities through 2027 there is an increasing role played by the US executive and national security state, but otherwise the key decision makers remain the lab executives. The board of OpenAI is understood to have entirely lost control of the firm, with little visibility over internal developments.

To an extent this makes sense considering the very short timelines within the scenario. The nature of bureaucratic corporate structures means that the lab CEO, if they desired to, cou...

Posts

Wikitag Contributions

Comments