As a father of two very young daughters (2 years old and 2 months old), I can really appreciate this. As someone with a background in computational neuroscience and some linguistics/NLP/ML/AI, I've loved watching them grow and making educated guesses about what sorts of computations could be going on inside their little brains at each developmental stage.
From the earliest days, when they can't even focus on our faces, hard as they try (I can tell what you're trying to do, superior coliculus and fusiform face area; you can do it!), to later on when they're walking and talking (still working on that theory of mind, though).
Language development has been especially fun to watch. Early on, they love just staring at your mouth as you enunciate the various phonemic sequences of what will become their native language. As they become more aware, you can see them start to comprehend when you use simple sentences to narrate things within their field of attention. And they definitely learn to understand more complex language long before they can talk. Patterns built upon patterns, just like deep transformer models, yet still quite different.
When my daughter began to pronounce words, we started pausing intermittently while reading her favorite books or singing familiar songs, and we would have her complete the last word of each line. I couldn't help but think of how large language models are often trained to perform next-token prediction in a similar way. Although, it's clear that the human brain has some sort of extra bias that makes it easier to memorize songs and poetry than prose.
And it's funny how trying to talk to babies reveals just how much of our adult-level world model we assume when we communicate. Once, when our oldest was trying to use a sippy cup on her own for the first time, we saw her putting it to her mouth like we did but failing to get any water. To help her out, I told her to lift the bottom of her cup to get at the water. She then proceeded to lift the entire cup above her head, which surprisingly did not help her. (Eventually she got it.)
For all their temporary limitations, it's clear that there is a lot going on inside babies' heads. You can learn a lot about the human brain and cognitive algorithms and biases by studying them carefully. It's certainly the cutest way to do so.
One thing I forgot to mention that I meant to say: My oldest can now speak English pretty fluently, using articles and adjectives, even some compound sentences; she just has first and second person pronouns mixed up.
When she says, "Go in your bed," she's referring to her own bed. Or when she says, "Give me the ball," she means that she is giving it to us.
This sort of mistake makes sense when you consider that she learned English by listening to her parents narrate things from our own perspective. Every time we said, "Thank you," she was giving us something, so now that's what she says when she gives us something. Every time we used first person pronouns, she correctly inferred that we were referring to ourselves (i.e., not her), and every time we used second person pronouns, we were referring to her.
When she speaks, it sounds like she's telling us what she predicts we would say, rather than what we would expect her to say as a proper conversational partner.
I've heard that babies start out life not just egocentric, but actually unable to distinguish the rest of the universe from themselves. Our youngest is still at that stage. When she is sad, it's because the universe is sad. Mommy and Daddy are not individuals out there operating independently in an external world; they're just phenomena that the universe generates to make everything better.
For our oldest, she knows that we are different people, but she seems to see language as a process of narrating everything that happens. Sentences are things we build together, rather than a way for different people to share their own perspectives with each other. "I/me/my" is always said about the other person, and "you/your" is always said about her.
For GPT-3, I feel it's the same way. Language is something it generates and predicts, not a conversation it participates in from its own perspective.
To correct this In our daughter, we say something like, "No, say, 'I pooped in my potty,'" if she says, "You pooped in your potty."
For a large language model, how would we get it to understand that it is separate from us, with its own limitations in understanding, which are different from our own limitations in understanding? Is that something we even want?
Mine (3.5 yrs) says "show me" when she wants to show something to her mom or me. Usually with an exasperated sigh between trying to describe the thing and just deciding we need to come see it.
[as our oldest sees it] Sentences are things we build together, rather than a way for different people to share their own perspectives with each other.
I've gone through a huge growth arc as an adult in recognizing the extent to which (especially in really good conversations) sentences are things we build together. Not that we don't have different perspectives, but when conversation is really flowing, it makes way more sense to view it as "our collective mind is thinking" and not "I am transmitting information to you, then getting information back" etc.
(When we're more at odds with someone, whether adversarial or just conflict with a loved one, it can be more like the transmit mode, and sometimes (tho not always) it seems to work best if we can get into the co-thinking mode again. Though there's not a hack for that—it's a deep trust-dancing puzzle!)
The 2nd half of this video is about this collective mind thing: https://youtu.be/G3vcXZPlsDc
Curated.
A couple years ago I curated "Growing Independence" as an interesting look into how children grow in agency, That post was about Jeff teaching his ~5 year olds to start making independent decisions. Some of my reasons then:
I think there is actually something fairly important about the topic of raising kids, which is relevant to more common LessWrong themes. First, there is a sense in which raising a family is one of the core things humanity is about. Many LW folk don't seem to have kids, and part of me is worried that all our philosophy and strategy is sort of missing something important if we don't have a background sense of "what raising kids is like" subtly informing our judgments.
There is also a sense in which this post is about "how to raise an agent", which I think ties pretty directly into core LW themes. I felt like reading the article fit into my overall worldview that includes robust agents and rationality and learning to think independently
This post is focused on very young children, which highlights a somewhat different set of ways children learn. But, I appreciate the throughline that I see drawn from this post, through Jeff's post, to the various LessWrong posts about how to be a grown up agent who goes out into the world, figures out what they want, and figures out how to achieve it.
In the first weeks, they learn something new every day in the first weeks They- at least if you pay attention like Jason. Research seems to show that there are distinct jumps in the development, though with all the incremental progress, they might be harder to spot. This has entered popular knowledge as "wonder weeks." I think in your description, there might be a hint here:
The first time I helped her sit up, at a few months old, the look on her face was wonder and amazement. I’m sitting up! How did that happen? I didn’t know that could happen!
For our sons, a phase transition that I remember distinctly was depth perception. One night they would be highly irritated start gazing into the distance as if focusing on something distant. It was really easy to observe by taking steps left and right with them on the arm.
Something else that is hidden in the post is the perception of themself as an actor:
he has servants to feed, clothe, and bathe her, to carry her from room to room
I think small babies don't realize that they are an actor distinct from their caretakers. They move their arms up - and leave the ground - by their parents picking them up, of course, but they don't have the concept of an actor yet. And when they do realize that it is often stressful too, they try a lot of things to figure out when it works and when not.
much of human motivation is curiosity and self-actualization
And status? I recall when my kid was having trouble learning to crawl, we showed him a toy baby that could crawl by itself, and he really hated it, bawling when he saw it start to crawl and then afterwards whenever he saw it. (Granted there could be other explanations for this behavior besides "status". We ended up helping to make the learning easier by putting him on a slight decline, which worked very well.)
We run races, climb mountains, compose ballads, peer through telescopes. These things don’t put food on one’s table, a shirt on one’s back, or a roof over one’s head. That’s not why we do them. We do them in order to be fully human and fully alive.
I notice that all of these can be explained by status seeking as well. (At least on the individual level. I guess there's still the question of why we award status for these particular things.)
It makes sense that these would be intertwined. Humans evolved to utilize our intelligence to find and make use of new opportunities; people who do this successfully bring more value to the family / tribe; so naturally we would evolve to find successful curious behavior (i.e. courage, adventure, etc) high-status in others, at the same time as intrinsically desirable in ourselves.
This was very sweet and reinforced my desire to have a child (a topic I'm chewing on at the moment).
In On Evolved Values, Robin Hanson argues that we will evolve to have more abstract desires to have children.
Can you describe what feeds into this desire? Where do you feel it (if you feel it anywhere at all)? What has influenced this desire in the past?
Love this! Having my first child really reinforced by views that IQ needs constant outlets to explore and express, and thus that the frame of the environment determines what types of skills the IQ puts effort into solidifying and reinforcing. I look at my little gradient descent machine scooting down the stairs, and I think to myself, "What a wonderful world."
Thanks for the drive down memory lane!
It reminds me of the day I saw my daughter discover her hands were attached to her body and in her control. She pounded the high chair tray till she discovered pounding her newly found hands hurt.
I never thought that you could learn induction at such a young age. You've made me shed a tear (':
At a certain point, she had learned that hands are good for holding things, but she would only grasp a toy if you literally put it in her hand.
Newborn babies do in fact have a grip-reflex: this does not need to be learnt initially. Put anything in the palm of a newborn and they will grasp it. I think there is a tendancy for this reflex to be lost prior to re-learning how to grasp things consciously.
Really enjoyed this. The human need for challenge and more than the fulfilment of immediate needs/ wants is an argument I've made to friends many times but I'd never thought of infants as such a good example of this.
[I]f children were properly engineered, they would be born limbless, and the arms and legs would grow in as they were needed.
Some cultures turn their babies into limbless larva by binding them tightly in cloth. Now I understand why.
Like the post a lot!
Further to
When the mere struggle for survival does not provide enough of [challenge ...] [...] we invent it for ourselves: through games and sports, through travel, through storytelling, through math and science. We run races, climb mountains, compose ballads, peer through telescopes.
I think a dark side is noteworthy here too: How once all is 'good' in our life, we are stunningly good at finding new as-if-life-threatening problems even if from the outside we'd have judged these purest luxury issues barely worth thinking about.
Maybe it depends a bit on the personal character - some manage to create somewhat more positively looking artificial challenges of the type you mention, and maybe others hurt themselves more by 'identifying' the new problems & crises.
I'm a father of four sons myself, and I'm very happy about this write-up and its lens on observation of learning of the growing child. Everything must be learned, and it happens stage by stage. Each stage builds on top of previous stages and in many cases, there are very many small steps that can be observed.
As someone who doesn’t have kids, but wants them one day, this was a really insightful piece. Thank you!
I miss this time period with my daughter.
She's now 12 and there are new and exciting things for her and I to learn, but the continuous drumbeat of new discoveries has slowed way down.
To be fed from a spoon requires three distinct skills. First, you must open your mouth when the spoon approaches. Second, you must close your mouth around the spoon once it has fully entered the mouth (and not before). Third, you must swallow the food that remains in your mouth after the spoon is withdrawn, rather than spitting it out.
I know this because my daughter, who is six months old, did not possess these skills two months ago. She learned rapidly—ah, to possess the neuroplasticity of the young—but there was a brief period when she literally did not know how to be spoon-fed. (Note that I have not begun to describe the skills required to feed oneself with a spoon, in part because she has not yet acquired them.)
The first lesson I have received from fatherhood is that everything must be learned, or very nearly everything. Babies are born with a very small number of reflexes and instincts: to suck on whatever enters their mouth, to “root” around on mother’s chest to find the nipple, to cry when they are uncomfortable. Everything else is a mental step in a long, upward climb.
In the first weeks, both the parents and the child are focused on digestion. The infant is essentially an alimentary canal with arms and legs. (The limbs are superfluous and indeed get in the way more than they help; if children were properly engineered, they would be born limbless, and the arms and legs would grow in as they were needed.) The infant is learning to nurse, to sleep, and to poop, and that’s about all they do for a little while. Yes, they even need to learn how to poop, or at least that’s how the nurse in the maternity ward explained her grunting sounds and puzzled expression.
All capabilities come in the tiniest increments. At first she did not know what her hands were for, and was more likely to scratch her own face than to do anything useful with them. After a few months she started to realize that hands could grasp things, but not how to do so. She would pinch and grip randomly when something was under them. Just feeling. She started to hold the bottle when I fed her, but she made every conceivable mistake in doing so. She would put her hands on top of the bottle, rather than on the bottom or sides. (Gravity has to be learned.) She would try to balance it on her knuckles, to comic effect. She would get a grip on the bottle, then move her hands and lose her grip while drinking. She would hold the bottle very close to the nipple, giving her no leverage to lift it. She would squeeze it between her wrists instead of using her palms and fingers. (She still does that, actually, and she’s gotten remarkably good at it.)
At a certain point, she had learned that hands are good for holding things, but she would only grasp a toy if you literally put it in her hand. She would do nothing purposeful with it, not even look at it, and then drop it randomly, unconsciously. Later, if you held a toy out to her, she would stretch her arms out and reach for it. But she would not yet reach for a toy that was nearby on the ground. Then one day she started reaching for those toys, too. This was a triumph: her first sign of pursuing an object of her choosing, rather than one which was (literally) handed to her.
Gross motor skills, like fine ones, have also come by degrees. At first she could push herself half an inch or so along the ground by flailing her legs, not always intentionally. Then it was intentional. Quickly she realized that by combining half-inch scoots, she could travel a longer distance—maybe multiple inches. At that point she was mobile, barely. For a brief time she would crawl towards a toy she wanted, but only if the toy was within a foot or two of her. Soon she grasped the inductive argument: if I can scoot N inches, then I can scoot N+1 inches. From then on her range was unlimited. (Although she has yet to get up on all fours, and she crawls by dragging her belly along the ground, commando style.)
The crawling, and the grabbing, lead me to the other lesson I have received so far, which is that much of human motivation is curiosity and self-actualization, not mere comfort and pleasure—even at this very early stage. She has not yet said her first word, and yet already she seems driven by an insatiable desire to explore—to explore both the world and her own abilities. She delights in her toys but is not content with them; she wants to move beyond the delimited rectangle of her play mat, with its smooth, round objects of wood and plastic in bright solid colors. She seeks the world beyond the mat: to touch the carpet and the curtains, to crawl behind and underneath the chair, to handle a grown-up cup, to pull clothes out of the drawer, to grab at hair and glasses and clothing, to eat the tag hanging underneath the sofa.
She crawls towards objects of desire, but not all her motion is directed at a tangible goal in the environment. She climbs over cushions, or up a foam-block incline, or up my chest, apparently for the fun of doing so. The first time I helped her sit up, at a few months old, the look on her face was wonder and amazement. I’m sitting up! How did that happen? I didn’t know that could happen! Now when I offer a hand, she doesn’t just sit up: she stands. Sitting is for three-month-olds. She stands: wobbling, swaying, shaking, sometimes almost collapsing into a seated position, usually getting back up with the slightest tug. The look on her face is still one of exhilaration, every time.
If her utility function were purely based on physical comfort and pleasure, most of this would be inexplicable. She already has the cushiest life imaginable: she has servants to feed, clothe, and bathe her, to carry her from room to room, to soothe her to sleep, even to wipe her bottom; her slightest whim or discomfort is attended to quickly if she but makes it known. If the goal of life were to relax and take it easy, she would already be at the pinnacle, with nowhere to go.
But—as she reminds me, with every striving crawl across the room, with every curious coo at a strange new object—relaxation is not the goal of life. Not of hers, nor of mine, nor of humanity’s. Our lives are not complete without challenge, adventure, play, and curiosity. When the mere struggle for survival does not provide enough of that—and it has not, since hunter-gatherer times—we invent it for ourselves: through games and sports, through travel, through storytelling, through math and science. We run races, climb mountains, compose ballads, peer through telescopes. These things don’t put food on one’s table, a shirt on one’s back, or a roof over one’s head. That’s not why we do them. We do them in order to be fully human and fully alive. And so does she, even if, for now, she is climbing sofa cushions instead of mountains, and peering at a set of plastic measuring spoons from the kitchen rather than at the cosmos.
Daughter, that’s what you’ve taught me, just in your first six months. I only hope I can ever teach you nearly as much.