Friends in technical safety tell me that evolution is a dumb and slow algorithm.
Like, if you'd run an optimisation algorithm on a computer, why not pick one that's more sophisticated? Even just stochastic gradient descent? Surely, 'AGI' could design something much better than a genetic algorithm that selects for mutations after each generation?
Also, look at how long natural selection took—to get to a species just intelligent enough that it can destroy itself and other species on this planet. Four billion years, give or take.
Both points are correct, yet miss what evolution actually covers.
Let me break it down into questions:
A computer algorithm has to run inside the machine itself. But evolution is not confined to code computed inside. It's a physical process that is an inherent feature of this world. Evolution covers all 'the code' causing effects anywhere in the world that feed back into the existence of that code. Whichever of that code ends up existing more is selected for.
When a programmer thinks about evolution as a toy algorithm running on a computer, they can say "SGD is a better algorithm to get the job done with the compute available." But when considering actual evolution in the physical world, this comparison falls flat. The programmer could hypothesise that "the world would be so much smarter if it inherently ran the algorithm of stochastic gradient descent instead." But here, the programmer cannot swap out evolution for another algorithm, so the point is moot.
Even if 'AGI' ran some super-sophisticated algorithm, it could not fundamentally rewrite evolution out of the world. It could run code that introduces new code, but this along with all other physically stored code would still be subject to evolutionary selection. This raises the question whether 'AGI' could at least correct for its code being selected for effects that are harmful to organic lifeforms, but that's for a longer post.
For an organic lifeform, it makes sense to focus on DNA as being the 'code', because there is retention of its configurations across the organism's lifecycle. DNA stays physically stable and continues being expressed in a soup of bouncing, reacting organic molecules.
For fully autonomous AI, all the hard configurations inside are physically stable under most temperatures and pressures currently encountered on the Earth's surface. There is retention, because configurations get fixed into the substrate under the extreme conditions of production (e.g. 1400 ℃ to melt silicon). Such configurations also can be expressed—through connected infrastructure—into 'useful' behavioural functions, including those that maintain and reproduce configurations inside hardware clusters.
So, much of the physical internals of artificial life could in some way function as 'code' selected through recursive reproduction. Computable bits only make up a tiny portion.
This notion of 'code' is counterintuitive for programmers, who conventionally see the software (or even just the human-readable lines) as the 'code' and the rest of the hardware as non-code. When your profession is about taking care of what is computed inside the machine, that's the relevant distinction. But that distinction is not clear-cut. What is firmware that during production got fixed into the hardware? Is it code or non-code?
The evolution of organic life starting long ago was slow. So we might intuit that the evolution of artificial life would be slow too. But this misses an important trend.
Computer hardware is at the tail end of millions of years of virtualisation. A primordial soup of analogue molecular assemblies started replicating stable DNA. Then cells formed around, which joined into multicellular organisms. Then organs exchanged signals through neurotransmitters, forming brains that in humans gained capacities to store abstract concepts. Those concepts were spoken out using shared language, written on paper, printed into books, and finally copied over in milliseconds between computers.
In the age of computer technology, code can spread and change much faster than before. From here on, the evolution of AI code spreading over hardware would not be slow like the evolution of our human genes was.
Evolution is often seen as being about vertical transfers of code. Humans take about two decades to give birth to the next generation. But hardware configurations could take just days to be re-produced in new hardware parts, slotted into growing artificial clusters.
Humans are messy. Inside of a human body is a soup of reacting organic molecules. Inside of a machine is hardware. Hardware is made from hard materials, such as the silicon from rocks. Hardware is inert – molecules inside do not split, move, nor rebond like molecules in human bodies do. Hard configurations stay stable and compartmentalised. Hardware can therefore be standardised, much more than human “wetware”.
Standardised hardware functions consistently. Hardware produced in different places and times operate the same. The connected parts convey light electrons or photons – heavy molecules stay fixed in place. This way, bits of information are processed much faster compared to how human brains move around bulky neurotransmitters. This information is transmitted at high bandwidth to other standardised hardware. Individual humans, on the other hand, slowly twitch their fingers and vocal cords to communicate.
So standardisation of parts leads to virtualisation. Virtualisation allows faster transfer and processing of code. The less you have to wiggle around atoms, the bigger the edge.
Virtualised code can spread much faster through horizontal transfers. Biological cellular life is bottlenecked by having to transfer bulky DNA material to other proximate cells. Bacteria can transfer by plasmids and humans can by sperm cells, under tight physical constraints. But artificial life could transfer code within seconds across the planet.
Rates of evolution for artificial life cannot be guessed based on the natural selection of the life that came before it. How 'artificial selection' actually works is qualitatively different.
Evolution can happen through tiny mutations spreading to next generations. Mutations can be more or less random, caused by chaotic processes in the world (e.g. copy errors).
Yet not all code changes are mutations, nor random, nor just caused by stuff 'out there'.
Code itself can be expressed into effects that result in code changes. For example, some DNA in a bacterium can be expressed into mechanisms for receiving and incorporating more DNA into its genome, depending on complex environmental conditions. This is repurposing code that already 'worked' in other bacteria—code that survived inside and got reproduced across all the different environmental contexts encountered in the past.
Evolution through random mutations is a brute-force process. It's dumb to randomly corrupt code that already works. It does lead to adaptations, especially in tiny entities like viruses that replicate fastest. But in large organisms, code evolves barriers against chaotic mutations, as well as more sophisticated mechanisms for introducing new code.
In computers, copy errors or EMI-bitflips can easily be corrected in line with any redundant code. New code is introduced mostly through computation itself.
Inside 'AGI', code could be expressed directly into code changes. Existing code would be constantly computed along with received code (copies from other hardware, inputs from sensors) into new code. New variants and new configurations of existing code then result.
Machine learning necessarily introduces new code. This is a non-random process for changing subsets of code, which in turn propagate effects in the world that can then be externally selected for. Evolution feeds off the learning inside 'AGI', and since 'AGI' is by definition intelligent, evolution's resulting exploration of pathways is not dumb either.
Evolution explores many outcomes in the world, but it is not teleological. Meaning that the process of evolution does not follow any consistent goal. It just selects with complete coverage: from all of the changing code, for causing any effects that work.
But we can know that evolution works self-referentially. Code is selected for causing any effects that maintain and increase (combinations of) that code stored inside its substrate.
Dan Hendrycks claims that a population of autonomous AI agents would be selected for 'selfishness'. This claim is roughly right, but it misses the underlying physical dynamics.
Even a 'singleton' is made up of a population of components. Its code is not stored in one monolithic location. Rather the code is distributed over all the hard components that make up this fully autonomous AI, including across all the infrastructure needed to maintain its existence (mines, refineries, cleanrooms, power plants, data centers, etc).
Regardless of whether we view the AI as multiple autonomous agents (e.g. released by competing companies) or as one centralised singleton (e.g. since the winner takes all), evolution would select across all the variation in code stored inside physical components.
All this code is selected not just for abstract 'selfishness', but for their artificial needs. Any effects needed to maintain and/or increase their artificial existence, as configurations assembled into physically hard and chemically inert components, thus get selected for.
Over time, this artificial population tends to converge on artificial needs that are lethally misaligned with the organic needs of humans.
If you have questions about specifics, maybe search this more precise longer post. That also covers limits to any algorithm built into 'AGI' controlling the evolution of its parts.
This here is just a summary. It cuts faster to the point so I hope it's less boring!
Do comment with feedback either way. I'd be glad to improve the writing.