I recently came across the short essay below by George Eliot, the male pen name of Victorian author Mary Ann Evans. It's quite remarkable in its prescience of current AI debates, particularly how long the "servant-master" relationship can persist between humans and their machines.

What's striking for me is that I'd read several of her novels such as Middlemarch and Adam Bede, but nothing about these fairly typical Victorian works ever hinted that she was thinking about the kinds of questions raised in this essay.

Shadows Of The Coming Race

My friend Trost, who is no optimist as to the state of the universe hitherto, but is confident that at some future period within the duration of the solar system, ours will be the best of all possible worlds--a hope which I always honour as a sign of beneficent qualities--my friend Trost always tries to keep up my spirits under the sight of the extremely unpleasant and disfiguring work by which many of our fellow-creatures have to get their bread, with the assurance that "all this will soon be done by machinery." But he sometimes neutralises the consolation by extending it over so large an area of human labour, and insisting so impressively on the quantity of energy which will thus be set free for loftier purposes, that I am tempted to desire an occasional famine of invention in the coming ages, lest the humbler kinds of work should be entirely nullified while there are still left some men and women who are not fit for the highest.

Especially, when one considers the perfunctory way in which some of the most exalted tasks are already executed by those who are understood to be educated for them, there rises a fearful vision of the human race evolving machinery which will by-and-by throw itself fatally out of work. When, in the Bank of England, I see a wondrously delicate machine for testing sovereigns, a shrewd implacable little steel Rhadamanthus that, once the coins are delivered up to it, lifts and balances each in turn for the fraction of an instant, finds it wanting or sufficient, and dismisses it to right or left with rigorous justice; when I am told of micrometers and thermopiles and tasimeters which deal physically with the invisible, the impalpable, and the unimaginable; of cunning wires and wheels and pointing needles which will register your and my quickness so as to exclude flattering opinion; of a machine for drawing the right conclusion, which will doubtless by-and-by be improved into an automaton for finding true premises; of a microphone which detects the cadence of the fly's foot on the ceiling, and may be expected presently to discriminate the noises of our various follies as they soliloquise or converse in our brains--my mind seeming too small for these things, I get a little out of it, like an unfortunate savage too suddenly brought face to face with civilisation, and I exclaim--

"Am I already in the shadow of the Coming Race? and will the creatures who are to transcend and finally supersede us be steely organisms, giving out the effluvia of the laboratory, and performing with infallible exactness more than everything that we have performed with a slovenly approximativeness and self-defeating inaccuracy?"

"But," says Trost, treating me with cautious mildness on hearing me vent this raving notion, "you forget that these wonder-workers are the slaves of our race, need our tendance and regulation, obey the mandates of our consciousness, and are only deaf and dumb bringers of reports which we decipher and make use of. They are simply extensions of the human organism, so to speak, limbs immeasurably more powerful, ever more subtle finger-tips, ever more mastery over the invisibly great and the invisibly small. Each new machine needs a new appliance of human skill to construct it, new devices to feed it with material, and often keener-edged faculties to note its registrations or performances. How then can machines supersede us?--they depend upon us. When we cease, they cease."

"I am not so sure of that," said I, getting back into my mind, and becoming rather wilful in consequence. "If, as I have heard you contend, machines as they are more and more perfected will require less and less of tendance, how do I know that they may not be ultimately made to carry, or may not in themselves evolve, conditions of self-supply, self-repair, and reproduction, and not only do all the mighty and subtle work possible on this planet better than we could do it, but with the immense advantage of banishing from the earth's atmosphere screaming consciousnesses which, in our comparatively clumsy race, make an intolerable noise and fuss to each other about every petty ant-like performance, looking on at all work only as it were to spring a rattle here or blow a trumpet there, with a ridiculous sense of being effective? I for my part cannot see any reason why a sufficiently penetrating thinker, who can see his way through a thousand years or so, should not conceive a parliament of machines, in which the manners were excellent and the motions infallible in logic: one honourable instrument, a remote descendant of the Voltaic family, might discharge a powerful current (entirely without animosity) on an honourable instrument opposite, of more upstart origin, but belonging to the ancient edge-tool race which we already at Sheffield see paring thick iron as if it were mellow cheese--by this unerringly directed discharge operating on movements corresponding to what we call Estimates, and by necessary mechanical consequence on movements corresponding to what we call the Funds, which with a vain analogy we sometimes speak of as "sensitive." For every machine would be perfectly educated, that is to say, would have the suitable molecular adjustments, which would act not the less infallibly for being free from the fussy accompaniment of that consciousness to which our prejudice gives a supreme governing rank, when in truth it is an idle parasite on the grand sequence of things."

"Nothing of the sort!" returned Trost, getting angry, and judging it kind to treat me with some severity; "what you have heard me say is, that our race will and must act as a nervous centre to the utmost development of mechanical processes: the subtly refined powers of machines will react in producing more subtly refined thinking processes which will occupy the minds set free from grosser labour. Say, for example, that all the scavengers work of London were done, so far as human attention is concerned, by the occasional pressure of a brass button (as in the ringing of an electric bell), you will then have a multitude of brains set free for the exquisite enjoyment of dealing with the exact sequences and high speculations supplied and prompted by the delicate machines which yield a response to the fixed stars, and give readings of the spiral vortices fundamentally concerned in the production of epic poems or great judicial harangues. So far from mankind being thrown out of work according to your notion," concluded Trost, with a peculiar nasal note of scorn, "if it were not for your incurable dilettanteism in science as in all other things--if you had once understood the action of any delicate machine--you would perceive that the sequences it carries throughout the realm of phenomena would require many generations, perhaps aeons, of understandings considerably stronger than yours, to exhaust the store of work it lays open."

"Precisely," said I, with a meekness which I felt was praiseworthy; "it is the feebleness of my capacity, bringing me nearer than you to the human average, that perhaps enables me to imagine certain results better than you can. Doubtless the very fishes of our rivers, gullible as they look, and slow as they are to be rightly convinced in another order of facts, form fewer false expectations about each other than we should form about them if we were in a position of somewhat fuller intercourse with their species; for even as it is we have continually to be surprised that they do not rise to our carefully selected bait. Take me then as a sort of reflective and experienced carp; but do not estimate the justice of my ideas by my facial expression."

"Pooh!" says Trost (We are on very intimate terms.)

"Naturally," I persisted, "it is less easy to you than to me to imagine our race transcended and superseded, since the more energy a being is possessed of, the harder it must be for him to conceive his own death. But I, from the point of view of a reflective carp, can easily imagine myself and my congeners dispensed with in the frame of things and giving way not only to a superior but a vastly different kind of Entity. What I would ask you is, to show me why, since each new invention casts a new light along the pathway of discovery, and each new combination or structure brings into play more conditions than its inventor foresaw, there should not at length be a machine of such high mechanical and chemical powers that it would find and assimilate the material to supply its own waste, and then by a further evolution of internal molecular movements reproduce itself by some process of fission or budding. This last stage having been reached, either by man's contrivance or as an unforeseen result, one sees that the process of natural selection must drive men altogether out of the field; for they will long before have begun to sink into the miserable condition of those unhappy characters in fable who, having demons or djinns at their beck, and being obliged to supply them with work, found too much of everything done in too short a time. What demons so potent as molecular movements, none the less tremendously potent for not carrying the futile cargo of a consciousness screeching irrelevantly, like a fowl tied head downmost to the saddle of a swift horseman? Under such uncomfortable circumstances our race will have diminished with the diminishing call on their energies, and by the time that the self-repairing and reproducing machines arise, all but a few of the rare inventors, calculators, and speculators will have become pale, pulpy, and cretinous from fatty or other degeneration, and behold around them a scanty hydrocephalous offspring. As to the breed of the ingenious and intellectual, their nervous systems will at last have been overwrought in following the molecular revelations of the immensely more powerful unconscious race, and they will naturally, as the less energetic combinations of movement, subside like the flame of a candle in the sunlight Thus the feebler race, whose corporeal adjustments happened to be accompanied with a maniacal consciousness which imagined itself moving its mover, will have vanished, as all less adapted existences do before the fittest--i.e., the existence composed of the most persistent groups of movements and the most capable of incorporating new groups in harmonious relation. Who--if our consciousness is, as I have been given to understand, a mere stumbling of our organisms on their way to unconscious perfection--who shall say that those fittest existences will not be found along the track of what we call inorganic combinations, which will carry on the most elaborate processes as mutely and painlessly as we are now told that the minerals are metamorphosing themselves continually in the dark laboratory of the earth's crust? Thus this planet may be filled with beings who will be blind and deaf as the inmost rock, yet will execute changes as delicate and complicated as those of human language and all the intricate web of what we call its effects, without sensitive impression, without sensitive impulse: there may be, let us say, mute orations, mute rhapsodies, mute discussions, and no consciousness there even to enjoy the silence."

"Absurd!" grumbled Trost.

"The supposition is logical," said I. "It is well argued from the premises."

"Whose premises?" cried Trost, turning on me with some fierceness. "You don't mean to call them mine, I hope."

"Heaven forbid! They seem to be flying about in the air with other germs, and have found a sort of nidus among my melancholy fancies. Nobody really holds them. They bear the same relation to real belief as walking on the head for a show does to running away from an explosion or walking fast to catch the train."

New Comment
5 comments, sorted by Click to highlight new comments since:

Definitely surprising of George Eliot, but seems like mostly echoing & enlarging on Erewhon a few years before.

This is really remarkable! Thanks for posting, I don't know when I would have seen it otherwise.

Partway through, I Googled a couple passages to quickly confirm that it wasn't some "modern blog post in an old-timey style" -- it feels more specific and prescient to me than I remember Erewhon being (though Gwern's comment makes me think I should go back and look again, or check Darwin Among the Machines).

Evans' predictions on Trost's arguments and overall vibe are also impressive, IMO. (E.g. "if it were not for your incurable dilettanteism in science as in all other things--if you had once understood the action of any delicate machine..." ~= "actually training models gives you the insight that AI risk isn't a thing.") I wonder how much Evans-vs.-Trost-style conversation was already going on.

Interesting how this part turned out:

how do I know that they may not be ultimately made to carry, or may not in themselves evolve, conditions of self-supply, self-repair, and reproduction

It doesn't seem like Evans (or others at the time?) anticipated that instead of individual machines reproducing like humans, they could just run the factories / mines / etc. that produce them. Which is an odd oversight, since by that time machines were certainly used in factories to make more machines -- it seems like the "reproduction via external construction" prediction should have been a lot easier to make than the "reasoning machines" prediction.

"Heaven forbid! They seem to be flying about in the air with other germs, and have found a sort of nidus among my melancholy fancies. Nobody really holds them. They bear the same relation to real belief as walking on the head for a show does to running away from an explosion or walking fast to catch the train."


Wow. I love this short story by G. Eliot, and it's nice to find it here. Just a question, maybe you can help...

I have a translation to Spanish (not yet published), and I have a doubt with this (to me cryptic reference): 

by this unerringly directed discharge operating on movements corresponding to what we call Estimates, and by necessary mechanical consequence on movements corresponding to what we call the Funds, which with a vain analogy we sometimes speak of as "sensitive."

Do you know what she means/alludes with "Estimates" "Funds"?

BTW, My take on the origin of this story is that it is a reference to Bulwer-Lytton's novel Vril (The Coming Race, 1871)

She starts that paragraph as a humorous updating of British Parliament to the politics of a future machine age, imaging speeches replaced by arcs of electricity in a Parliament of machines. "Estimates" is Parliamentary jargon for approved budget numbers; "Funds" are the actual pools or accounts of money, like "sinking fund" or the "government fund" for the Royal Society etc.

I am not quite sure what she means by analogy to sensitive: she is comparing "sensitive" in the animal meaning, of having electrified nerves which sense the world and which enable reactions or reflexes, to another use of the word in presumably British politics, but I don't know of any Victorian political use of 'sensitive' and can't quickly find one - even if it is used heavily in contemporary political speech to talk about particularly important or controversial things. Perhaps it was used that way back then too? Or she just thinks the word's application is obvious even if it is not a standard jargon.

If so, then her point would seem to be that while we talk in vague metaphors about contemporary human organizations being 'sensitive' or having 'nervous systems', we flatter our incompetent, inefficient, self-propagandizing "idle parasite" organizations by ascribing much more competence to them than they deserve. In contrast, a superior future machine government would literally have nerves conducting electricity back and forth among all its constituents at the speed of light and be sensitive. The governing machines would reach out with electricity and directly adjust a budget estimate in light of new information, and then the corresponding bank accounts would automatically increase/decrease, and downstream, machines would execute their purpose without any delay or hesitation, "blind and deaf" with "no consciousness" at all, simply doing things with a superhuman rapidity and precision. (You would no more have idle parasite Parliamentary machines than you have idle parasite 'aristocratic gears' in a steam engine.)

Thank you very much for your comments... and clues.