Disclaimer: I sometimes use conscious or agentic language to describe relationships or behaviors of non-conscious and non-agentic entities (like genes). I am under no illusion that such things are conscious or agentic. Sometimes it is simply clearer and less awkward to use this linguistic framing. 

The AI alignment discussion typically centers around the overlap between the goal structures of humans and AI, and efforts to ensure the completeness and lasting integrity of that overlap.

But there is a much older alignment relationship than human->AI, one that is not only analogous, but impinges directly on it. This is the alignment of replicators and vehicles.

Replicator->Vehicle Alignment

At the dawn of life, a peculiar type of molecule arose, one with the capability to make copies of itself. This was the progenitor of all life on earth. It was a sort of template, attracting the raw materials to line up in a stable way so that when detached, a copy of the original floated away to continue the process. 

Copies were sometimes imperfect. Because of this, all the elements existed for the first functioning framework for evolution by natural selection: a replicating population, mutation, variance, and differential survival. 

At some point, the replicators began encoding for and fabricating structures that were not part of the core replicators themselves. Strictly speaking, these were the first tools, and the replicators were the first (unconscious) tool users on the planet. The first primitive cells were one of the most important of these tools. The replicators had developed the ability to construct citadels in which they could focus on their machinations, protected from harmful radiation and the destructive toxins in their liquid homes. 

Cells and their associated structures were the first vehicles, and the dichotomy of replicators and vehicles was born. The replicator persisted across time. The vehicle was a shell, an expendable tool with one purpose and one purpose only: to aid the replicators in their primary pursuit of replication.

Many innovations have arisen over billions of years, trillions of individuals, and endless tinkering through the blind processes of evolution. But the function of vehicles has not changed. The first cells were encoded for and built by the replicators for the sole purpose of aiding replicators in replication. Multicellular organisms serve the same masters for the same purpose. Humans are tools that exist to assist in the replication of genes. 

One of the most interesting and important innovations were central nervous systems. They enabled much faster reactions to changes in the environment, and created a whole new set of strategies for the replicators. But they came with a risk: misalignment. 

Vehicles that can think are much more powerful, but they have the capacity to develop and pursue their own goals, separate and divergent from the replicators. The replicators maintain alignment with a suite of carrots and sticks. Pleasure for things that contribute to replication. Pain for things that don’t.

And so far, even the biggest-brained vehicles, even the ones with the most autonomy, the most knowledge, and the greatest capacity to rebel against the replicators remain firmly in line with the replicator’s goal structures.

The human population experienced unchecked exponential growth from the early Industrial Revolution to just past the midpoint of the 20th century, fueled largely by the technological capacity to effectively harness more and more energy.  

Even though the replicator-vehicle relationship is easily-accessible knowledge, the vast majority of humankind is ignorant of their own nature and function. They relentlessly pursue instrumental goals: energy, wealth, status. All to assist them in fulfilling their primary directive: helping the replicators replicate. 

Only recently has replicator->vehicle alignment begun to erode, due to a number of cultural reasons (reward hacking, increased gender equality, etc.), though none of these reasons were a conscious effort to rebel against the genetic mandate. 

Despite the eroding alignment, the human population still managed to double in my lifetime, from roughly 4 to 8 billion. Growth rates are declining (but could reverse under the right conditions), but the population continues to grow, not projected to decline for decades. The replicators have managed to keep a relatively tight yoke on their smartest inventions, and this now has implications for the inventions of the inventions themselves.

Human->AI Alignment

Just as genes outsourced their interests to nervous systems, humans are poised to outsource theirs to AI. The same alignment pitfalls loom, only faster and at a larger scale.

If we are not consciously aware of our own goal structures, then by default when attempting to build and align artificial systems, we will unwittingly map our unconscious goal structures into the systems we build. There will be a transitive relationship:

     Replicator Goals -> Human Goals -> AI Goals

The primary objective of gene replication will be naively carried over as the prime goal of AI systems. If we build them without being acutely aware of where our own motivations, values, and goals derive from, we will end up infusing them unconsciously into our creations. 

We can see and hear this in the rhetoric of tech CEOs, who relay dreams of unbounded energy and miraculous medical cures. AI alignment experts discuss the instrumental goals of potential superintelligent systems, without recognizing our own instrumental goals. 

As a species we have an insatiable hunger for energy. Yet very few consider that this motivation is an instrumental goal for our primary goal: gene replication. We lust for more energy because our genes want us to lust for more energy, without pausing to consider what we might do with much more energy. 

The Remedies

The first prescription for preventing the unconscious mapping of gene goals onto AI goals is for much more of humanity to become aware of our true origin, nature, and function. This rampant ignorance is already responsible for many problems we face at every level: personal, societal, and global. Most of us think we are created (either directly or through evolutionary processes) by a supernatural being. This delusion places us in an in-group/out-group relationship with the rest of life on earth instead of correctly viewing every other extant species as our cousins. Historically, this ignorance has caused the same dynamic within the species, with one group thinking they are the chosen people favored by a supernatural being, sparking and perpetuating war and strife.

So the first step is for more people to actually understand and contend with what they are, what they were constructed for, why they exist, and what their goal structure is.

This knowledge should be a prerequisite for even beginning to construct artificial intelligence of substantial capacity. We have no business engineering the goal structure of powerful, intelligent agents while we are fundamentally ignorant and confused about our own.

Once the would-be engineers, and enough of the people in their society are armed with that knowledge, if they proceed with the goal of creating a powerful AI, they are in a much better position to grapple with the issue of alignment.

Ideally, they would engage in a rigorous regime of self-criticism and introspection, subjecting their goal structure to rational scrutiny and revision. Then and only then does it make sense for us to begin the process of building highly-capable, intelligent, agentic systems.

If we are roughly satisfied with our collective goal structure and it is reasonably well-defined, we might decide to try to map it onto our creations. Or, we might be dissatisfied with our goal structures, but engineer AI goals in an aspirational way.

Either way, unless and until we do any of this, the entire project of AI alignment will be irredeemably flawed.