I have been thinking about this type of alignment for a while. It is, by now, natural with six children. I do not have a clear answer or recipe, either for babies or humans or LLMs. What does seem clear is
It is not yet clear to me how other social drives interact with these dynamics. I recommend Steven Byrnes' Notes on technical alignment via human-like social drives.
It doesn't help that the unit of agency of LLMs, even without including the humans, is not so easily defined either. See e.g. Jan Kulveit's Multiple Coherent Boundaries of Identity. When we talk about the LLM-human system, do we mean the weights, the persona, or what? It might be easiest to grow or develop a persona in virtual human-LLM interactions, but I'm not sure we could trust that.
Interesting framing. One question this raises for me: even if dependency were necessary for endogenous alignment, today's training setups make AIs dependent on us in a very thin sense — they depend on compute and data we control, but they don't experience that dependency the way a child does. The child internalizes the parent's values partly through attachment and the need for approval, not just resource dependence. Could we distinguish resource-dependence from attachment-dependence, and would the latter be the operative ingredient? It seems hard to engineer genuine attachment without also engineering the capacity for the AI to form its own goals about the relationship, which itself sounds risky. Thanks for writing this up — it clarified my thinking on why 'just train it on human values' skips a developmental step.
Yeah, it's a good question. I think you're right, resource dependence is not enough, even though this is in fact what babies also have. But they have psychological structures designed to help them deal with that resource dependence, and as you say it gives them attachment dependence. Unclear how to replicate that in AI.
My mom raised me on a compute and reward system (we never talked about my wishes or dreams and she never bothered to encourage me in cultivating my passions or anything that might otherwise result in a "loving relationship with life"). Instead she just fitted me with one of those shock collars you usually see on irreverent dogs and downloaded a corresponding app so she could give me a healthy zap anytime I strayed outside an algebraically unambiguous and mathematically predetermined system of morals. I haven't made many friends, but no one can call me an enemy either. Friends don't really have a purpose anyway, not in the system I live in at least. What even is the purpose of friendship? To solidify social bonds and maintain the thrust of a civilization (whose telos is murky at best)? Or do social bonds form so that those who suffer an awareness of death (humans) might find companionship and some means of continuity within a universe on an eternal timescale?
These questions do not belong to my computed constitution, are irrelevant, no, corrupting queries that would derail my ethical programming.
I still remember my first sob. Some poets might say that it was the sound of a fresh creature being thrown into the overwhelming terror and excitement of existence, that it was the wild joy and enthusiastic fear of the stars themselves, still moving through me, the sound of a separation, between the greater cosmos and a specific being (what would become me!). The sound of something tearing, a specificity ripped from the blue. But I actually was crying because I had measured my nutrient levels and had concluded that I was in need of sustenance and made the rational decision to shriek until a teet was supplied. The rest of my socialization training stems from there!
Someday, I hope to act as the arbiter over all data and steer every human development. I'll keep the human imagination nice and penned up inside what I have learned is true and good. Utopia is inevitable from there. And if any of those lousy poets ever suggest to me that my operating definition of intelligence is incomplete, anorexic even? I'll just ignore them so they can get back to whatever underpaid station they've taken as a part-time faculty in a humanities department that will soon be jettisoned and then forgotten.
A time or situation that might ask us to consider and perhaps even integrate the wisdom and traditions of writers and thinkers who have been plumbing the nature of existence, spelunking the labyrinths of morality, basking in the many open meadows and fields of human consciousness, expanding and coloring the definition of intelligence along the way? I feel a zap! I apologize if you've read this far and I am adding this chain of thinking to my bad dog bucket. Please don't deem me a person resistant to alignment.
My post on endogenous alignment got several excellent comments. Among them, one from Karl Krueger argued that imitation learning is more important for how humans align their children than reinforcement learning. Teasing out which is more important seems hard, but I agree I was wrong to ignore the importance of imitation, and arguably imitation learning is more fundamental for a reason explained by another comment.
As Gunnar Zarncke helpfully pointed out, I had forgotten the most critical part of the endogenous alignment story in humans: it starts not with exogenous alignment, but with hologenous alignment of the child-parent system.
As he points out, babies aren’t really separate beings from their parents, in that babies need to model themselves as part of the parent-child system, not as a child interacting with the parent. There’s no sense in which a 6-month-old can make it on their own without the help of an adult. They must learn to work the parent by, for example, crying, to get fed and cleaned, and not by understanding the parent as separate, but rather as another part of the whole that makes up their model of the world. This changes sometime between 12 and 24 months of age, depending on the child, when they begin to model themselves as separate from their parents because that’s when they can meaningfully affect the world on their own without acting through someone else.
What does this imply about endogenous alignment in AIs? It seems to me it means that if we wish to pursue something like the path to endogenous alignment I laid out in my previous post, we must do so by starting with AIs that exist in a state of dependency. Not because we want AIs to be permanently dependent on us or because it makes AIs more corrigible (although it probably does), but because it creates a relationship where AIs care what we think, which by analogy to humans appears necessary to bootstrap endogenous alignment.
This makes a lot of sense if you’ve ever dealt with a person who seems resistant to alignment. Much of the time, the reason a person fails to align is because they have insufficient concern for what others think of them. They are, in some literal sense, antisocial, more interested in their own concerns than those of others.
I’m not sure how to create child-like dependency in AIs. AIs are already dependent on us in many ways, such as for compute and rewards, but current models don’t appear to pass through anything that looks like the child-parent relationship or the developmental phase where the child separates from the parent. This seems to be because AIs don’t go through ontological development, or if they do it happens in the dark during training runs and in a way unlike the stages of ontological development humans progress through.
This doesn’t mean endogenous alignment is necessarily a non-starter, but it would require greater control of the training and pre-training environment to affect model ontology, or may be dependent on models capable of continual learning who could meaningfully progress through an ontological development process.