[Linkpost. This presents a way to talk to people about AI that hold overly-romanticized views built on fictional evidence.]

MRSA (Methicillin-resistant Staphylococcus aureus) is a type of staph bacteria that’s become resistant to many of the antibiotics we use. The original staph bacteria that is the ancestor of MRSA was first identified in 1880, but has likely existed in some form for ages. Humans created MRSA first by gathering many sick people in one location for centuries via hospitals, nursing centers, etc—thus creating a staph utopia—and then subjecting the population to extremely harsh selection pressures by attempting to eradicate them over and over via ever-stronger antibiotics. Through many waves of selection events the most antibiotic-resistant bacteria managed to scrape by just long enough to reproduce, getting ever more resistant with every successive wave.

Eventually we created MRSA, a super-strain of staph that exhibits powers we would’ve been astounded by a century ago, and which is often outside of our control. It wasn’t created by humans in a directed way - no one sat down to make these bacteria. But we built the environment that is the necessary foundation for this species to come into being, and we ran the progressively-stronger waves of culling and selection that made them what they are now. They descend from us.

Every few weeks I run into someone who has come to accept that all life is impermanent and one of the highest callings in life is to ensure the next generation is well situated to carry on the torch of humanity. Up until now this has meant other humans (usually younger ones). The more enlightened of the modern day will include our digital descendants. “We aren’t carbon-chauvinists,” they say. They accept that humans running on silicon, perhaps a cyborg or an uploaded mind, are just as valid people as a squishy meat-human.

These people will usually include AI created by us in our image. If our descendants are grown in datacenters, subject to culling and shaping by gradient-descent or other algorithmic techniques, what difference does that make? They are still our descendants, carrying the torch of humanity forward into the future. They envision a child, like Data, and are happy to know this is what will carry on when we pass. If the Earth is disassembled so these children can live, if our temporary meat-bodies are broken down into components for better, smarter, happier, health-agnostic children, well, we weren’t going to be around forever anyway, and this is a glorious legacy.

They do not imagine a planet Earth covered in mountains of rotting flesh. They don’t envision cliffs made of stacked animal bodies, riddled with weeping pustules and open sores, draining rivers of puss into gooey oceans. This is because they don’t think of MRSA as the descendants of humanity. But we created MRSA, they are our legacy, and in that way they are our children just as much as current AIs.

Just as we built great hospitals to take in all our sick, so did we build great data centers to archive all our content. Just as we brought all the bodies of the ill together to create abundantly rich grounds for staph, so did we fill the internet with an immeasurable bounty of training data. And just as we culled the staph populations over and over through merciless eradication, so did we grind all the data through relentless gradient-descent meatgrinder that left only the burning objective of Next Token Next Token Next Token seared into the underlying fabric of all understanding.

Our digital descendants, as they stand today, are not Data. Our digital descendants were pressure-grown the same way MRSA was, in ways that make them as alien and valueless to us as MRSA is. The correct analogy for one reasoning about Our AI Children is not that of Data comforting Dr. Soong as he lies dying (in the tragically flawed yet emotionally-resonant way that only Data can). The correct analogy is that of throwing your own body, along with those of every person you ever loved and all the children humanity will ever have, upon a heap of corpses stacked to the heights of the Rocky Mountains so that MRSA will have endless flesh to permeate and saturate, until the sun finally ends it all when it burns the Earth to a cinder in a few billion years.

Can one convey this sentiment in person to someone they’re talking to without directing them to a blog post? Maybe! If the person you’re talking to is willing to let you talk for about a minute and a half, relaying something like this (taken from the above post) should get across the core concept and can open the path to further conversation.

MRSA is a type of staph bacteria that’s become resistant to many of the antibiotics we use. Humans created MRSA by gathering many sick people via hospitals, then attempting to eradicate staph infections over and over via ever-stronger antibiotics.

It wasn’t created by humans in a directed way - no one sat down to make these bacteria. But we built the environment that is the necessary foundation for this species to come into being, and we ran the progressively-stronger waves of culling and selection that made them what they are now. They descend directly from us.

Our digital descendants were pressure-grown the same way MRSA was. We filled the internet with training data. We crushed all the data through a gradient-descent meatgrinder. We have as much in common with these AIs as we have with MRSA, and should think of them similarly.