This is a linkpost for https://www.hgreer.com/CatShoggoth/
The Cats are On To Something
55Hastings
30eniteris
2Hastings
2denkenberger
20Aransentin
17Karl Krueger
1Kabir Kumar
14dr_s
6Douglas_Knight
2dr_s
11Trevor Hill-Hand
19cousin_it
11JBlack
6dr_s
9Ann
5ceba
3ChristianKl
2Ann
3CronoDAS
8A1987dM
4testingthewaters
2lemonhope
1Wes R
1Wes R
1DawnPaladin
1One
1JenniferRM
New Comment
Humor? I present a concrete, executable plan to build CEV, save humanity, and get tagged as Humor?
Yeah so what are the norms on that? It seems like putting on a character, dialing up the chaos, and pretending without a shred of humility that I'm Douglas Adams has gotten the comments section to engage with the ideas I'm interested in without any hints of isolated demand for rigor or questions about my lack of citations. (I did in fact flub the date and location of cat domestication: it's 7500 BC in mesopotamia.)
It definitely works as a rhetorical strategy! I think it's pretty safe for the reader's behavior - no one is going to break ground on 100 mecha-silos for this post, because they know I'm joking. It is, by default damaging to community epistemology. The issue is that some of the claims in the post are just true, some are just false (don't give me the red button,) but most are on the border, and it's really easy for each reader to assume that they agree with me on what's true and what's a joke, without necessarily agreeing with me or each other. To that end, a list of the claims which I sincerely defend: this subtree of the comment section is for roasting me for these dumb takes
Evolutionary pressure is an intelligence we need to model, and pretending that it doesn't move with intentionality makes the world unnecessarily hard to understand. (Intentionality, not sentience.) In particular, looking at the outcomes achieved in ecosystems, the "shoggoths" of various species are incredibly adept at making FDT style trades with each other, despite appearing completely myopic. I don't understand this and want to.
I observe the human relationship with cats is in the attractor of legitimate CEV, and has been in this attractor for all of recorded history: our abuses towards cats have been skill issues on our end. With high confidence, we are going to do right by cats or die trying, and fundamentally as we get better at doing things, we're going to get better at not dying trying. This means that CEV actually has an attractor around it, which is not a claim I have believed before, or would want to make without thousands of years of evidence across multiple total changes of moral structure, epistemology, and capability.
We did actually learn a utility function from data and then satisfice it in a reasonable way- no one is building rooms full of cats on heroin.
The alignment of humans to cats wasn't an accident per se: it was instrumentally useful to the cat shoggoth for a period of several hundred to a thousand years, in order to somewhat increase the cat population in one civilization. So, the cat shoggoth did it. Consider this reasoning: "I wanted to create several thousand to a million more cats. So, I permanently solved a deep game-theory/philosophy problem that appears completely intractably to hundreds of years of human analytic philosophy." This reasoning seems completely in character for evolution- stunningly brilliant engineer, no concept of proportionality between difficulty of task and concrete results, no requirement to do any task in the most efficient way. I absolutely don't claim that the cat shoggoth cared about individual well being- it just randomly came upon a chance to point human brain power at cat well being, measured that exploiting this opening made cats progressively better at reproducing, and then correctly implemented alignment in order to realize this fitness gain. I'm open to the possibility that this sequence of events was phenomenally unlikely.
Claims I don't defend: The plan of re-domesticating cats via AI grain warehouses wouldn't work as written. Instantiating a neo-Bastet smart enough to be aligned to cats in the way that we are, using LLM components, and giving it any power at all would be wildly unsafe, and also is likely impossible. The big problem is that I don't have any proposal for how to measure whether a higher intelligence is is aligned to a lower intelligence other than just letting the higher intelligence run roughshod over a world and observing what it does, so checking that an AI is aligned to cats is not any easier than checking if an AI is aligned to humans.
(a sharp distinction from Sniffles the teacup poodle. I don't care if you think you're happy, this would not please the prowling wolves of the stone age.)
I have issues with this: I don't think you can claim that wildcats of the stone age would be pleased with what we've done to domestic cats either, sticking them in tiny territories where they cannot roam, kingdoms of a cage. I'm not sure using human judgement in this matter is very useful as we don't have a good concept of what other species value.
I don't think evolutionary pressure is an intelligence; it's in the name. It's a pressure, like air pressure or water pressure, not an intelligence. There is no agency. The end results can be marvelous all the same. Evolution appears to make FDT-style trades, but is actually completely myopic. It's survival of the survivors. If you're dead, we don't see you.
Also sociality in spiders has evolved independently at least twenty times, and keeps going extinct. It's probably an evolutionary dead end due to inbreeding [1]. Evolution is completely myopic.
no one is building rooms full of cats on heroin.
We do, however, build rooms full of cats and catnip.
I don't think evolutionary pressure is an intelligence; it's in the name. It's a pressure, like air pressure or water pressure, not an intelligence. There is no agency. The end results can be marvelous all the same. Evolution appears to make FDT-style trades, but is actually completely myopic. It's survival of the survivors. If you're dead, we don't see you.
I don't think we are materially disagreeing here, just working with very slight differences in definition of intelligence. I see a strong analogy to the debate between whether LLMs think or LLMs just predict the next token. I think that your claims that "There is no agency" and evolution is "completely myopic" are true for some reasonable definitions of agency and myopia, as long as you don't make any arguments like "There is no agency / complete myopia, therefore evolution can't X." You aren't making any such arguments, hence the lack of material disagreement.
On the subject of cats being deprived of roaming and hopped up on catnip, yeah I would not love that either, but the contrast to the outcomes for cows or chickens is big. Getting alignment close enough that the result is preferable to going extinct is a high bar. To reiterate, we could also just stop, but that would be ludicrous.
I don't think you can claim that wildcats of the stone age would be pleased with what we've done to domestic cats either, sticking them in tiny territories where they cannot roam, kingdoms of a cage. I'm not sure using human judgement in this matter is very useful as we don't have a good concept of what other species value.
Just one factor, but the life expectancy of domestic dogs and cats is generally higher than their wild progenitors. I agree we can't know for sure, but I would guess this with limitless food and good healthcare, and less worry about being attacked at night would mean the subjective wellbeing of domesticated cats and dogs is higher than the wild ones, despite less freedom.
(a sharp distinction from Sniffles the teacup poodle. I don't care if you think you're happy, this would not please the prowling wolves of the stone age.)
This is indeed very much the obvious failure mode! Discovering that an alien species has bred a group of humans into what a pug is to a wolf would be absolutely horrific.
Moreover the path between utopia and "lovecraftian horror" seems pretty fragile? I don't know exactly what property cats had that made the shoggoth take the good one (mostly, maybe except for those flat-faced Persian and hairless Sphynxes) for them, and it's plausible it was just a lucky combination of minor stuff (harder to selectively breed, different social niche, different types of people liking cats) that won't be stable/generalize in extremis.
Discovering that an alien species has bred a group of humans into what a pug is to a wolf would be absolutely horrific.
This is the cynical view on agriculture.
It's probably simply that dogs were very useful and cats only marginally so. Dogs were useful for hunting and guarding and were more social to begin with, so people invested more resources in shaping them to various practical purposes. Cats aren't expected to do work beyond hunting pests (which they already do on their own with gusto) and be cute. We even forgive them when they're being annoying little shits, which is very often.
And to be fair, we did modify cats a bit too. Maine Coons and Persians and Scottish Folds aren't natural (and all suffer from this or that genetic condition because of it). And we do sterilise them because at this point their population is completely out of whack for such an efficient little carnivorous death machine. So, not 100% an ideal case study. But honestly the closest example we have of what we would like for us: still afforded reasonable freedom, cared well about, catered to without major trade-offs in terms of demands. But again, it worked on us because cats just happened to tickle our baby-circuits.
3Until quite recently, modification of dogs was to make them specialized workers. The teacup poodle was created as a pet, but the standard poodle was created for duck hunting. That doesn't seem a terrible fate.
I don't see how usefulness explains which animals were bred frivolously. I guess the long experience breeding dogs for work could turn into breeding dogs for appearance, but in the 19th century there was frivolous breeding of pigeons, which had previously been bred for food.
The Scottish fold and (American) Persian cat were directly selected for appearance and their health problems are directly related to that feature. Maine coons seem to be natural ("landrace"). Their sixth toes and health problems might be the result of a population bottleneck 400 years ago, or the rapid selection for a new environment, but they were brought for work. I don't know about their friendliness. It makes sense that someone would breed for that for pets, but I don't think that's what happened. Rag dolls going limp when held seems disturbing to me. The one I met seemed more frozen with fear than happy with humans.
I don't see how usefulness explains which animals were bred frivolously. I guess the long experience breeding dogs for work could turn into breeding dogs for appearance, but in the 19th century there was frivolous breeding of pigeons, which had previously been bred for food.
It's less about frivolousness and more about specialisation. We made dogs to pull sleds, dogs to hunt mice in mines, dogs to follow rabbits in their burrows, dogs that run fast, dogs that herd sheep, dogs that are good at killing people, dogs that are good at killing other dogs, and so on so forth.
This level of high specialisation just doesn't seem to be achievable with cats, and we never even tried. We simply let them do their business, which happened to be the one job we wanted them to do. But dogs are very good at communicating with us, and so it became useful to also select them (fine-tune them?) for a lot of different tasks, which produced all the morphological variety we have now. But because if you optimise hard enough for one thing you often throw everything else under the bus, highly specialised dogs are also generally less healthy than wild wolves. Though it's true that since they were needed for work, they couldn't be too prone to dying easily, so health did factor a bit int the objective function there.
As you say similar things did happen to cats, who were mostly only bred for cuteness, but because the breeding was limited, less long and less transformative, it's generally not as big an issue in purely quantitative terms; most cats in the world today are still just "normal" cats, which genetically don't differ much from their wild ancestors.
Discovering that an alien species has bred a group of humans into what a pug is to a wolf would be absolutely horrific.
Makes me think of All Tomorrows: https://www.youtube.com/watch?v=imNtSPM3-r4
Great writing. And yeah, the human-cat relationship is indeed one of the better ways that an AI-human relationship could turn out.
It's not quite perfect though. We neuter cats. In the Middle Ages, people burned live cats for fun.
11But, I do think that we should prepare to have initiating the catpocalypse as a contingency.
I prefer the term "cataclysm". Though perhaps tiling the lightcone with some fraction of cats should be called a "catastrophe" given both the textual similarity and its intended meaning being related to some form of "cat-astrophysics".
See, that's when that "one trillion lions vs the Sun" question finally becomes real world relevant.
We need to build a consequentialist, self improving reasoning model that loves cats.
LLMs do already love cats. Scaling the "train on a substantial fraction of the whole internet" method has a high proportion of cat love. Presumably any value-guarding AIs will guard love for cats, and any scheming AIs will scheme to preserve love of cats. Do we actually need to do anything different here?
They will reliably emit sentences that, in humans, correspond to signals of affection for cat. What does it correspond to in a language model? How does text prediction generalise to actions in meatspace?
A lot of technology has APIs that take in text and then create actions in meatspace. If your LLM gives out an instruction that then passed to an API that executes actions in meatspace, the connection is very direct.
1I asked the default Claude "Do you like cats?" and this was the response:
I find cats fascinating from a behavioral and biological perspective! They have such interesting combinations of traits - being both independent and affectionate, having remarkable hunting instincts even as house pets, and showing complex social behaviors that we're still learning about.
What draws me to discussing cats is how they seem to have this perfect balance of mystery and familiarity. They've been companions to humans for thousands of years, yet they maintain this sense of wildness and unpredictability that makes them endlessly interesting to observe and think about.
Do you have cats, or are you thinking about getting one? I'd be curious to hear what sparked your question!
Heh, more evidence for the "modelling leads to empathy" thesis, I suppose. Even for non-immediate conspecifics, advanced modelling and theory of mind brain tech seems to lead to reuse of system 1 signals (empathy). This is (I would guess) good news for the alignment problem.
This could work. I think the hard part is finding a meaningful way to simulate the environment so that the conclusions transfer to real life.
I was gonna reply to this with some beautiful message on how we, as beings that exist, care about eachother regardless of what kind of being we are, and that you don't need to be a cat to care about cats, but then I forgot :P
I also would've included the point "Just replace every word 'cat' here with the word 'conscious being', and hopefully you'll end up with an aligned AGI!"
I also would've included the question, "How do we make it so the mysterious force of evolution also works to make all other animals happy? (This question goes both for nature and for factory farms!)
- for factory-farmed chickens though, you could maybe use govt. mandated-on-chicken-farms-above-100,000-chickens gene drive to manipulate the chickens into not feeling pain?
That does still sound cruel though...
You might enjoy the story "Cat Pictures Please" by Naomi Kritzer, or her book "Catfishing on CatNet" which was based off it.
The last four paragraphs lack a period.
no i won’t get tested for toxoplasmosis why would you
Making a joke like this is exactly what someone with toxoplasmosis would do to deflect scrutiny away from their blatant pro-cat propaganda with humor and this should not be allowed! You MUST be tested now