This article addresses the interaction between the alignment of an artificial intelligence (AI) and the power balance between the AI and its handlers. It aims to clarify some definitions in a more general article, and to show that capability control and motivation control may have identical effects on the alignment...
AI Takeoff refers to “the process of an Artificial General Intelligence (AGI) going from a certain threshold of capability (often discussed as "human-level") to being super-intelligent and capable enough to control the fate of civilization”. Whether this process will be “soft” (i.e. manageably slow) or “hard” (i.e. fast) is key...
This article addresses the interaction between the alignment of an artificial intelligence (AI) and the power balance between the AI and its handlers. It aims to clarify some definitions in a more general article, and to show that capability control and motivation control may have identical effects on the alignment of an AI. In order to prevent confusion with more general uses of the terms “capability control” and “motivation control”, here I will use the terms “extrinsic alignment” and “intrinsic alignment”. These are probably not new concepts, and I apologize to the readers who already know them by different names or using a different formalism; I haven’t found a clear description elsewhere.
For the... (read 852 more words →)
AI Takeoff refers to “the process of an Artificial General Intelligence (AGI) going from a certain threshold of capability (often discussed as "human-level") to being super-intelligent and capable enough to control the fate of civilization”.
Whether this process will be “soft” (i.e. manageably slow) or “hard” (i.e. fast) is key to determining survivability of AGI, so estimating take-off speeds is important to design strategies to manage the advent of AGI. Current methods to predict this speed are based on qualitative arguments, averaging the opinions of experts or using very fundamental principles such as bioanchors (summarized here).
This essay makes the following points: