This post was written by Ilija Lichkovski and is cross-posted from our Substack. Kindly read the description of this sequence to understand the context in which this was written.
The founders of automation startup Mechanize published a blogpost arguing in favor of the view that “the tech tree is discovered, not forged”:
Humanity is often imagined to be like a ship captain, with the ability to chart our course, navigate away from storms, and select our destination. Yet this view is wrong.
To a first approximation, the future course of civilization has already been fixed, predetermined by hard physical constraints combined with unavoidable economic incentives. Whether we like it or not, humanity will develop roughly the same technologies, in roughly the same order, in roughly the same way, regardless of what choices we make now.
Upon making their case, the authors conclude that the march to the full automation of the economy through AI is inevitable, and that one can only choose to accelerate such a transition, never oppose it. Although I provide a summary below, I would encourage you to read this short piece before proceeding.
I think it’s important to read it because it is as captivating as it is fundamentally misguided — herein I share an alternative that can yield us a more realistic, messier, and yet still empowering, understanding of technological history.
There is a temptation to find the determinism espoused by the authors as fatalistic and even morally repugnant, as it minimizes human agency and propagates a narrative that our actions are inconsequential. I find these narratives to be disempowering as we approach one of the crucial technological eras in our civilization’s history.
Irrespective of that, however, the idea of technological determinism should not be discounted. It is, in many ways, true that technological progress is built upon the primitives of physics and the economic incentives of the civilization in question.
The Mechanize authors marshal two compelling observations. Firstly, simultaneous discovery is common — the Hall-Héroult process for aluminum smelting was discovered in 1886 by Charles Martin Hall in the United States and Paul Héroult in France, independently. Alexander Graham Bell and Elisha Gray filed telephone patents on the same day, hours apart. Secondly, isolated civilizations converge on similar technologies — the Aztecs and Spanish had both independently developed intensive agriculture, hierarchical states with bureaucracies, and monumental architecture, despite 10,000 years of separation.
These patterns suggest that when prerequisites align, invention quickly follows. Technologies emerge “spontaneously” when necessary conditions are met. The authors conclude that humanity is “like a roaring stream flowing into a valley, following the path of least resistance,” where attempts to steer development “will only delay the inevitable, not prevent us from reaching the valley floor.”
Where I differ from the Mechanize authors is the extent to which I assume a maximalist interpretation of tech tree determinism. This is not an essay about determinism versus free will. Fundamentally, I argue that inevitability is incompatible with the sensitivity of history to small perturbations, often enacted by small groups or even individuals. Specifically, my critique is predicated upon two foundations: the omnipresence of power laws that govern historical outcomes, and the fact that the technological tree is embedded into larger social contexts which complicate its implications.
The largest oversight in the determinist framework is the assumption that influence is uniformly distributed, across time or across people. It is not. A handful of events and people have outsized effects — power laws govern historical outcomes just as they govern everything else. Most of the work in group projects is performed by a couple of people, most scientific progress is ushered in by a handful of scientists, and a handful of moments in history determine outcomes for decades or centuries after. The “branching factors” of different moments in history vary wildly — there exist certain decisive nodes from which radically different futures can follow.
Consider Stanislav Petrov, a lieutenant colonel at the Soviet Air Defense Forces. On September 26, 1983, Soviet early-warning systems reported five incoming American ICBMs. Protocol demanded Petrov report the attack up the chain of command, which would have triggered full nuclear retaliation. Instead, Petrov judged it a false alarm and declined to report it. He was correct — it was a satellite malfunction. Had Stanislav Petrov followed protocol, he would likely have drawn the curtains on advanced civilization on planet Earth. He is hardly the lone case — another Soviet officer, Vasili Arkhipov, during the Cuban Missile Crisis, was the sole vote against launching a nuclear torpedo when his submarine came under depth charge attack. These men lived during moments of extraordinarily high “entropy”, and thankfully rose to the moment. History is abundant with opportunities for the slightest differences to yield the most significant consequences.
Even scientific progress as a whole, in fact, has been anything but inevitable. A compelling case has been made that a large driving factor for the Scientific Revolution is the spread of Catholicism in Europe, which disrupted kinship networks (giving rise to role-based association, followed by universities) and cemented moral universalism as a fundamental cultural prior; which then was a cultural foundation for the Enlightenment, which then precipitated the Industrial Revolution and the Great Divergence. These empiricist foundations for the scientific revolution are by no means a default; rather, they are a historical abberation. Just as AI startup founders today make the case that total automation of the economy is inevitable, a peasant from the Dark Ages would be making the case that feudalism is the inevitable order of human affairs. Both will have been proven wrong by the stochasticities of history, where power laws dictate trajectories.
Moreover, the Mechanize authors claim the Aztecs and Spanish converged on similar technologies as evidence. The observations in their blogpost cherry-pick similarities while minimizing crucial differences. Surely Aztec societies used cotton and built monuments, yet European colonists commanded sea-faring empires while the Aztecs remained relatively localized to their continent, and the two civilizations differed in their capacity for organized violence (a kind of social technology). The convergence was superficial; the divergences were stark and crucial.
Fundamentally, the Mechanize authors are making a rough macroscopic argument (“humanity will develop roughly the same technologies, in roughly the same order, in roughly the same way”). It’s not necessarily an argument in favor of determinism, but rather in favor of the convergence of different trajectories. However; this is unsubstantiated — the rough assessment is insufficient to account for the sensitivity of history to singular events on the tail of the power law distribution. These are events that history hangs in the balance of, and once you stack many of them together across societies and technologies, you get more uncertainty about the future, not less. A few hundred year delay there, a socioeconomic hurdle there, and this multiplied over zillions of tech tree trajectories means that technological history diverges instead of converging. So we’re back to probabilities, and exponentially branching futures.
We can only be glad Stanislav Petrov and Vasili Arkhipov were not possessed by delusions of the inevitability of nuclear war.
There is undoubtedly a directionality and causality in the tech tree — we needed the transistor before the CPU. But that says nothing about key implementation details and chosen paths. Only a narrow subset of directions in the tech tree end up being pursued, and this is governed by academic incentives, industrial secrecy, and arbitrary historical contingencies. Perhaps even more importantly, the tech tree is conditioned and constrained by the civilizational context in which it is pursued.
For instance, high speed rail is technologically plausible, but the US is famously horrible at building it. The California High Speed Rail project is infamous for its costs ballooning into tens of billions of dollars with repeated delays. The downturn of nuclear energy itself is not to be neglected either — the foregone energy production over the decades due to nuclear phase-out is significant, and mostly due to a cultural motive. The US Navy has operated hundreds of nuclear reactors over 6,200+ reactor-years with zero accidents, while one Soviet accident due to institutional incompetence has radically altered public perception, even leading to outcomes as ridiculous as Germany demolishing nuclear capacity in the 2020s. The tech tree is almost always beholden to higher powers.
Even if the timelines above are too short, the fascinating example of movable type applies at a larger timescale. Movable type printing was invented in China centuries before Gutenberg’s press in the West, but it simply could not have the drastic implications due to the difficulty of casting type for thousands of characters. Instead, the printing press took off in Europe because of the nature of Latin type, which was more suitable with its fewer letters. In the following decades and centuries, Gutenberg’s press proved instrumental to the kind of technological progress that skyrocketed literacy and kickstarted the great divergence, despite its precursor technology having been developed elsewhere first. The outcomes of the same technology diverged entirely because of the context in which the technology was situated, and the effects reverberated across the entire tech tree. None of this was inevitable.
Superintelligence and the ensuing automation of the economy can prove to be sublime, or it could be catastrophic. The specifics, rather than the rough outline, determine this, because we are already on an exponential progress curve where the branching factor of decisions in this era of human history is astronomical. In this moment, fatalism about the direction of history is something is practically destructive too. There are drastically different outcomes possible with automation alone, to the extent that it’s doubtful that even what Mechanize is doing could be termed merely “accelerating” the outcome.
Instead, each contribution — from a fatalistic blogpost, to an RL environment sold to a top lab — shifts, ever so slightly, the distribution of probabilities over the next step in the future, of whether we maintain strong social trust and institutions during the AI transformation, whether we maintain human agency in the transition, and whether we preserve what we value about work and purpose.
These are not questions the tech tree answers for us. These are choices we make, at decisive nodes, where individual decisions and institutional quality determine which branch we take. The march to advanced AI is underway. But Stanislav Petrov showed us that at crucial moments at the tail of the power law distribution, what matters is not the inexorable march of technology, but human will and wisdom.