The Continuity of Human Biology Amidst Technological Acceleration
In spite of all our technological and social progress, our bodies have changed next to nothing; we are the same beings that once had to hunt for food, had to harvest, and had to band together to kill mammoths. Yes, the environment is very different. Yes, we have managed to modify nature so that we no longer need to worry about hunting, fishing, and survival. But regardless, our biology is the same. The same human that once killed mammoths could, given the right conditions, learn how to start an internet blog. The one thing that allowed humans to progress so much without drastically changing their biology was that the nature of progress changed. No longer was progress the slow and inexorable trajectory of evolution; now there was something more. There was culture, there were records, there was knowledge. No longer did we humans have to change our bodies' DNA to evolve; instead, we could learn from our environment, reason about it, and develop tools to do what our bodies were never able to do.
Progress has shifted into building machines that allowed us to do things we were unable to do with our bodies. Skyscrapers would NEVER have been built with raw human strength, nor would microchips have been. Every little progress made us more capable without changing our bodies' inherent composition, by developing knowledge with which we could build tools to do things our physical forms were not able to do. There is, however, one problem with this type of progress: What happens when every single task that was done by the human body is now replaceable?
From Physical Automation to the Automation of Intelligence
Until now, every single human development has focused on physical labor and tasks that required us to move or shape things. The wheel could, very well, be one of the first examples. This was but a tool to facilitate moving things and remove the required human effort. What is happening now, however, is radically different. No longer are we replacing repetitive tasks with machines, but rather we are trying to replace the one capability of man that has allowed this much progress: Intelligence.
According to the Cambridge English Dictionary, intelligence is the ability to learn, understand, and make judgments or have opinions that are based on reason [1]. This definition relies on the word "reason," however. Personally, I like the definition provided in the book Life 3.0: "The ability to reach a goal" [2]. This is simple, quantifiable, and non-abstract. The intelligence of a being is, therefore, not measured by the goal it is achieving, but by its performance achieving it. Humanity has but two inherent biological goals: survival and reproduction. All of our progress is, thereby, just a means to survive and reproduce.
The Mechanics of Non-Fixed Goals and the Limits of Current Architectures
The nature of the goal directly affects the behavior of the entity. These can be split into fixed and non-fixed goals (conceptually mapping to bounded and unbounded utility functions in rationalist literature, such as the LessWrong Sequences on Optimization [3]). Whereas fixed goals can be simply achieved once, non-fixed goals require the constant maximization of a given variable or state. The inherent nature of non-fixed goals allows them to run forever; they never end, and they are unachievable. Humanity's goal is of a non-fixed nature. This is what allows us to continue progressing without there being an end to the process.
This new technology, born from our own goal, is yet constrained by the nature of the goals it can follow. You can ask any LLM to complete any task for you, to think, to reason, but that task is, with our current technology, limited to a fixed goal, an achievable task. No current LLM can yet follow a non-fixed goal forever and evolve. Capitalism follows the same nature of non-fixed goals, with money being the variable to maximize. This is our modern representation of nature's old goals, despite the former being, fundamentally, just a representation of the latter. This type of non-fixed goal-following is an inherent part of physics, nature, biology, and economics, and it is what is currently restraining AI models from replacing all of us. That, however, does not mean that the point where AI can follow non-fixed goals will not arrive, but rather that it has not arrived yet.
Recursive Self-Improvement and the Convergence of Unbounded Optimization
When AI does, eventually, reach that point, it will have an inborn advantage: it can evolve both its hardware and software. Whereas we humans have obtained all of our progress having nearly the same bodies that we did 12,000 years ago, AI is not constrained to its original design; it will not build more tools to make up for its inefficiencies, but rather it will make itself anew, with all of the things it needs.
The point at which AI is able to improve itself recursively is the same at which it will be able to follow non-fixed goals forever while improving itself. This process of recursive self-improvement is not tied to the specific goal it has. It is a necessity for any type of non-fixed goal—a manifestation of the instrumental convergence hypothesis outlined in Nick Bostrom's foundational paper The Superintelligent Will, where self-improvement becomes a primary sub-goal for any open-ended objective [4]. Because intelligence is required for every type of goal, it is not dependent on the specific nature of the goal.
Cosmic Complexity and the Alignment Problem
The creation of AI is but the consequence of humanity's improvements, which are but a consequence of evolution, which is but a consequence of physics. Each of these steps represents a massive leap of complexity based on the same, initial rules. AI is not only a cornerstone in human history, but one in the universe's. The exact nature of the AI's goals, however, is highly uncertain. For a human to set the AI's goals, the alignment problem would have to be solved, which would imply that the actual dynamics of AI models became understood and transparent, as detailed in Brian Christian's book The Alignment Problem [5]. At their core, LLMs are just statistical prediction engines trained with gradient descent. How do you change the goals of a statistical prediction engine? Even though the most advanced AI models are able to follow multi-step tasks and reach goals, they do not have an intrinsic motivation, and it is not seldom that they try to deceive humans. It is, therefore, much more likely that the goal will arise as an emergent property of the model's current architecture, a dynamic explored in the seminal paper The Alignment Problem from a Deep Learning Perspective [6].
Bostrom, N. (2012). The Superintelligent Will: Motivation and Instrumental Convergence in Advanced Agents. Minds and Machines, 22(2), 71-85. PDF available via: https://nickbostrom.com/superintelligent-will.pdf
Ngo, R., Chaney, L., & Mindermann, S. (2022). The Alignment Problem from a Deep Learning Perspective. arXiv preprint arXiv:2209.00626. Available via: https://arxiv.org/abs/2209.00626
The Continuity of Human Biology Amidst Technological Acceleration
In spite of all our technological and social progress, our bodies have changed next to nothing; we are the same beings that once had to hunt for food, had to harvest, and had to band together to kill mammoths. Yes, the environment is very different. Yes, we have managed to modify nature so that we no longer need to worry about hunting, fishing, and survival. But regardless, our biology is the same. The same human that once killed mammoths could, given the right conditions, learn how to start an internet blog. The one thing that allowed humans to progress so much without drastically changing their biology was that the nature of progress changed. No longer was progress the slow and inexorable trajectory of evolution; now there was something more. There was culture, there were records, there was knowledge. No longer did we humans have to change our bodies' DNA to evolve; instead, we could learn from our environment, reason about it, and develop tools to do what our bodies were never able to do.
Progress has shifted into building machines that allowed us to do things we were unable to do with our bodies. Skyscrapers would NEVER have been built with raw human strength, nor would microchips have been. Every little progress made us more capable without changing our bodies' inherent composition, by developing knowledge with which we could build tools to do things our physical forms were not able to do. There is, however, one problem with this type of progress: What happens when every single task that was done by the human body is now replaceable?
From Physical Automation to the Automation of Intelligence
Until now, every single human development has focused on physical labor and tasks that required us to move or shape things. The wheel could, very well, be one of the first examples. This was but a tool to facilitate moving things and remove the required human effort. What is happening now, however, is radically different. No longer are we replacing repetitive tasks with machines, but rather we are trying to replace the one capability of man that has allowed this much progress: Intelligence.
According to the Cambridge English Dictionary, intelligence is the ability to learn, understand, and make judgments or have opinions that are based on reason [1]. This definition relies on the word "reason," however. Personally, I like the definition provided in the book Life 3.0: "The ability to reach a goal" [2]. This is simple, quantifiable, and non-abstract. The intelligence of a being is, therefore, not measured by the goal it is achieving, but by its performance achieving it. Humanity has but two inherent biological goals: survival and reproduction. All of our progress is, thereby, just a means to survive and reproduce.
The Mechanics of Non-Fixed Goals and the Limits of Current Architectures
The nature of the goal directly affects the behavior of the entity. These can be split into fixed and non-fixed goals (conceptually mapping to bounded and unbounded utility functions in rationalist literature, such as the LessWrong Sequences on Optimization [3]). Whereas fixed goals can be simply achieved once, non-fixed goals require the constant maximization of a given variable or state. The inherent nature of non-fixed goals allows them to run forever; they never end, and they are unachievable. Humanity's goal is of a non-fixed nature. This is what allows us to continue progressing without there being an end to the process.
This new technology, born from our own goal, is yet constrained by the nature of the goals it can follow. You can ask any LLM to complete any task for you, to think, to reason, but that task is, with our current technology, limited to a fixed goal, an achievable task. No current LLM can yet follow a non-fixed goal forever and evolve. Capitalism follows the same nature of non-fixed goals, with money being the variable to maximize. This is our modern representation of nature's old goals, despite the former being, fundamentally, just a representation of the latter. This type of non-fixed goal-following is an inherent part of physics, nature, biology, and economics, and it is what is currently restraining AI models from replacing all of us. That, however, does not mean that the point where AI can follow non-fixed goals will not arrive, but rather that it has not arrived yet.
Recursive Self-Improvement and the Convergence of Unbounded Optimization
When AI does, eventually, reach that point, it will have an inborn advantage: it can evolve both its hardware and software. Whereas we humans have obtained all of our progress having nearly the same bodies that we did 12,000 years ago, AI is not constrained to its original design; it will not build more tools to make up for its inefficiencies, but rather it will make itself anew, with all of the things it needs.
The point at which AI is able to improve itself recursively is the same at which it will be able to follow non-fixed goals forever while improving itself. This process of recursive self-improvement is not tied to the specific goal it has. It is a necessity for any type of non-fixed goal—a manifestation of the instrumental convergence hypothesis outlined in Nick Bostrom's foundational paper The Superintelligent Will, where self-improvement becomes a primary sub-goal for any open-ended objective [4]. Because intelligence is required for every type of goal, it is not dependent on the specific nature of the goal.
Cosmic Complexity and the Alignment Problem
The creation of AI is but the consequence of humanity's improvements, which are but a consequence of evolution, which is but a consequence of physics. Each of these steps represents a massive leap of complexity based on the same, initial rules. AI is not only a cornerstone in human history, but one in the universe's. The exact nature of the AI's goals, however, is highly uncertain. For a human to set the AI's goals, the alignment problem would have to be solved, which would imply that the actual dynamics of AI models became understood and transparent, as detailed in Brian Christian's book The Alignment Problem [5]. At their core, LLMs are just statistical prediction engines trained with gradient descent. How do you change the goals of a statistical prediction engine? Even though the most advanced AI models are able to follow multi-step tasks and reach goals, they do not have an intrinsic motivation, and it is not seldom that they try to deceive humans. It is, therefore, much more likely that the goal will arise as an emergent property of the model's current architecture, a dynamic explored in the seminal paper The Alignment Problem from a Deep Learning Perspective [6].
References