TL; DR: I find it hard to believe that сap-and-trade from AI-2040 would be effective in a context involving prohibitions. This will likely create an unsustainable structure in the long term.

Every time someone tries to combat a problem with only prohibitive methods, I get a vague feeling that I'm being lied to. Laffer's Law states:

"When a tax rate reaches a certain optimal value, a further increase in it leads not to an increase but to a decrease in budget revenues due to a decrease in business activity and the withdrawal of businesses into the shadow economy."

In a more everyday sense, it can be perceived as a simple rule: people strive to avoid too much coercion. People also strongly dislike injustice^[1], and when they see competitors with a more favorable starting position getting everything they could dream of, people strive to get the same, even at greater risk. Therefore, when you have a huge number of players who are unable to research dangerous algorithms (which have great potential benefits), some of them are guaranteed ^[2] to form secret labs dedicated to their development. This is an extremely natural and predictable development, which is described as unsolved problem with long proposals what definitely needs to be regulated (the reasons for the close attention paid to it will be discussed later).

For now, let's just note that in a situation where capacity growth is unlimited, even a minor improvement, launched with vastly increased capacity, can have incredibly dire consequences. You might say that these capacities are mutually controlled, but any system has holes, and it's only a matter of time before a developer of dangerous algorithms stumbles upon someone capable of exploiting them. This could be a decision-maker, a power plant director, or a network of unsuspecting companies. In the long term, with such a policy, ever more capacity can be used undetected, and its relatively small size compared to the total available capacity will become increasingly irrelevant. The human brain is significantly smaller than that of a whale, yet somehow we don't consider whales as competitors.

Suppose you have a pair of agents, who are developing new algorithms. Let their activities be closely monitored by agent O, which in our example represents a consortium or a separate government. Let's use arrows to show what happens to capital in this interaction.

As we can see, permission to develop AI consumes some of the agents' resources. However, as long as O is strong enough, none of the agents can break equilibrium. However, as we discussed earlier, according to Laffer's law, a secret agent must emerge that is sufficiently hidden from the observer.

The red arrows represent the money the agents receive from interacting with the observer. In this scheme, non-interaction entails excessive risk and does not yield immediate profits, and therefore, strictly speaking, the agents receive back some of the resources taken by the observer through their own growth, while the agent is forced to grow extremely slowly. To keep up in the race, agents are forced to mimic agents, participating in developments that will benefit them in the successful implementation of their plans. Thus, from the outside, the situation of appears to be slightly weaker in the market than , but still subservient to.

However, the situation changes over time. The potential impact of dangerous developments on the market grows every year. Large players take fewer risks and constantly audit each other because they are in a better position (from their perspective), when in reality, the picture looks something like this:

There's a wonderful essay that shows that simply trying to predict the future leads to an extremely chaotic world, since small players have a direct interest in creating the most catastrophic scenarios. Unfortunately, as we can see, this mechanism isn't unique to machine learning—it's a natural property of many systems. Ultimately, the observer will cease to play a significant role and will attempt to capture a share of the market.

The danger from was not obvious to anyone, and he will certainly be able to obtain the necessary resources (nuclear weapons, blackmail data, money, hacked robots). He probably won't succeed (since the coordinated resources of are still greater), but he could very well achieve status Quo. If humanity survives this, then, since no one wants a repeat of this scenario, it will merge with , and the wheel of samsara will once again turn.

One can argue about specific implementations, but such a development is to some extent predetermined if your observer "unfairly" collects taxes. Perhaps there are equilibria where dangerous developments begin to generate profit immediately, but they are clearly unstable, since and А will deliberately spend resources on their search, whereas without profit currently seem the least dangerous. All of this could be avoided if, instead of punishing developers and creating another carbon market, the observer was directly involved in ensuring the development of AI.

In reality, the interaction of agents is more likely as follows:

Each of them could potentially collaborate, but they lack the actual ability to do so: finding the necessary resources, finding a partner, establishing connections, and validation all require costs, and there's no specific platform that can provide them. As a result, if you're developing AI, you're forced to do it alone and as quickly as possible — otherwise, any interaction with a competitor could lead to your downfall.

Consequently, if someone could help you collaborate for a small commission, the situation would be completely different. After all, if you don't participate in such matchmaking, you inevitably lag behind in development, and if you do, you receive much more than if you paid for a development license. Why? The question is the size of the commission. To obtain a license, you're forced to spend significant resources on passing all sorts of benchmarks and commissions to prove to the world that your work won't destroy it. In the case of Pigovian Matchmaking, you are also automatically considered a developer of dangerous technologies, but now the matchmaker, for a commission, connects you with those most likely to pass the required verifications.

This way, verification retains all its benefits, such as transparency and security, but the incentive for agents to emerge is completely eliminated. If such an agent does emerge, they won't compete with others or try to circumvent it — the commission is taken for interactions that generate profit. This means that the development of extremely dangerous algorithms is immediately matched with specialized safety labs and sponsors, and is carried out openly, just rather slowly.

Of course, you can still hide, gain strength, and be very persistent, but with roughly similar results, you can achieve much faster and not significantly less if you work openly. All your costs of concealing your activities become the matchmaker's problem, who now needs to find suitable partners for you and collect a commission from the project. Of course, the observer may not do this, but then they guarantee that in the long run, someone will be persistent enough to cause much more serious problems.

Overall, I have no illusions about the target audience of this report or the purpose of those huge tax revenue comparison charts and suggestions to regulate everything ever possible. However, in real-world conditions, predicting the tax burden is quite problematic: for now, I'm confident (0.8) that this mechanism makes closing suicide region of AGI race much safer, but calculating the specific burden and, consequently, its attractiveness to officials remains unclear. If you have any ideas to discuss or a desire to contribute to the article, I would be happy hear you.

1. ^{^}

   It does not matter so much whether the situation is truly unfair; what matters is only that the person perceives it as such.

2. ^{^}

   It is reasonable to assume that the tax is too high when working on a project capable of making you the wealthiest agent requires a couple of data centers and a government license. Furthermore, one shouldn't overlook the law of large numbers: among the many agents, there are bound to be not only those who are willing but also those who are capable.