AI Takeoff refers to “the process of an Artificial General Intelligence (AGI) going from a certain threshold of capability (often discussed as "human-level") to being super-intelligent and capable enough to control the fate of civilization”.

Whether this process will be “soft” (i.e. manageably slow) or “hard” (i.e. fast) is key to determining survivability of AGI, so estimating take-off speeds is important to design strategies to manage the advent of AGI. Current methods to predict this speed are based on qualitative arguments, averaging the opinions of experts or using very fundamental principles such as bioanchors (summarized here).

This essay makes the following points:

Considering limiting factors may help predict AI take-off speed.
The speed of the first phase of the take-off process (develop superintelligence) may not be at all informative of the speed of the overall process (up to where AGI becomes capable enough to control the fate of civilization).
The methodologies that we should employ to control and survive the take-off process may have nothing to do with AI research.

What are limiting factors, and why do they matter?

The rate of any process is determined by its limiting factors. One can think of a limiting factor as the slowest step in a chain of production. To illustrate this concept, imagine a population of bacteria growing in a test tube. Their growth is the product of many chemical reactions happening inside the cells. Now consider a condition in which the slowest of these reactions is the ability of the bacteria to uptake glucose from the medium. What happens if we massively speed up this one reaction, for example by adding a ton of glucose to our test tube, or using genetic engineering to fill the bacterial membrane with glucose pumps? Will growth become infinite? No. Growth will increase until glucose uptake becomes non-limiting, which means that some other reaction down the chain is unable to process the large amounts of glucose that are being sent their way. If we want to predict how quickly our bacteria will grow after removing one limiting factor (glucose uptake), we must figure out which one will dominate the rate of the process next.

The speed of development of superintelligent AI won’t tell us much about the whole take-off process

The full take-off process requires an AGI to become super-intelligent&...

Posts

Wikitag Contributions

Comments