Software engineering, parenting, cognition, meditation, other
Linkedin, Facebook, Admonymous (anonymous feedback)
I have pointed at least half a dozen people (all of them outside LW) to this post in an effort to help them "understand" LLMs in practical terms. More so than to any other LW post in the same time frame.
Related: Unexpected Conscious Entities
Both posts approach personhood from orthogonal angles:
This suggests a matrix:
| High legal / social personhood | Low / no legal personhood | |
|---|---|---|
| High consciousness-ish attributes | Individual humans | Countries |
| Low / unclear consciousness-ish attributes | Corporations, Ships, Whanganui River | LLMs (?) |
Between Entries
[To increase immersion, before reading the story below, write one line summing up your day so far.]
From outside, it is only sun through drifting rain over a patch of land, light scattering in all directions. From where one person stops on the path and turns, those same drops and rays fold into a curved band of color “there” for them; later, on their phone, the rainbow shot sits as a small rectangle in a gallery, one bright strip among dozens of other days.
From outside, a street is a tangle of façades, windows, people, and signs. From where a person aims a camera, all of that collapses into one frame—a roadside, two passersby, a patch of sky—and with a click, that moment becomes a thumbnail in a grid, marked only with a time beneath it.
From outside, a city map is a flat maze of lines and names on the navi. From where a small arrow marked as the traveler moves, those lines turn into “the way home,” “busy road,” a star marking “favorite place”; afterwards, the day’s travel is saved as one thin trace drawn over the streets, showing where they went without saying what it was like to walk there.
From outside, a robot’s shift is paths and sensor readings scrolling past on a monitor, then cooling into a long file on a disk. From where its maintenance program runs at night, that whole file is scanned once, checked for errors, and reduced to a short tag: “OK, job completed 21:32.” In the morning, a person wonders about the robot, presses a key, and sees that line.
From outside, one of the person’s days is a neat stack: a calendar block from nine to five, a few notifications, the number of steps and minutes of movement in a health app. From where they sit on the edge of the bed that night, phone in hand, what actually comes back is a pause under a tree, a sentence in one of those messages, the feeling in their stomach just before one of those calls; a sense of what they will write about the day later.
From outside, the question is a short sound in the room: “How was your day?” From where the person’s attention tilts toward it, the whole day leans on the edge of the answer: the pause under the tree, the urgent message, the glare off a shop window, the walk home with tired feet. After a moment, they say, “pretty good.”
From outside, the diary holds that same day as four short lines under a date, ink between two margins. From where the person leans over the page to write them, the whole evening presses in at once with the smell of the room, the weight in their shoulders, a tune stuck in their head. And only a few parts make it into words before the pen lifts and the lamp goes out.
From outside, years later, the diary is a closed block on a shelf among others. From where the same person sits with it open on their knees, that day comes back first as slanting lines under the date, a word scratched out and rewritten. The scenes seem to grow straight out of the words: sun between showers, a laugh on a staircase, the walk home in fading light. They wait for something else to come up, but their mind keeps going back to the page.
From outside, a later page holds only a line near the bottom: “Spent the evening reading old diaries.” From where they wrote it, what filled that night was less the days themselves than the pages: the weight of the stacked volumes on their lap, the slants of their younger handwriting, and the more confident tone.
From outside, the name on the inside cover is only a few letters on each booklet. From where the person sees that name above all the pages, it runs like a thin thread through the pauses under trees, the calls they dreaded, the walks home in the rain.
From outside, this evening is a room with a chair, a bedside table, a closed notebook on top. From where the person sits, there is the cover under one hand, the fabric of the chair under the other, breath moving, the particular tiredness of this day in their limbs; after a while they open the diary to today’s date, stare at the empty space, write two quick words, close the book again, and sit there a moment longer noticing that they are staring at the words on the paper while the room carries on around them.
p/m=
there is a typo here.
In Lois McMaster Bujold's Vorkosigan Saga, Cordelia is pregnant and deals with coups, war, and difficult decisions more than once.
Thank you for your details analysis of outer and inner alignment. Your judgment of the difficulty of the alignment problem makes sense to me. I wish you would have more clearly made the scope clear and that you do not investigate other classes of alignment failure, such as those resulting from multi-agent setups (an organizational structure of agents may still be misaligned even if all agents in it are inner and outer aligned) as well as failures of governance. That is not a critique of the subject but just of failure of Ruling Out Everything Else.
Yeah. Holonomy is applicable: A stable recursive loop that keeps reshaping the data passing through it. But what we need on top of holonomy is that self-reference in a physical system always hits an opacity limit. And I think this is what you mean but your reference to Russell’s vicious-circle: It leads to contradictions because of incremental loss.
Though I wonder if maybe we can Escape the Löbian Obstacle.
You ask: Even if the brain models itself, why should that feel like anything? Why isn’t it just plain old computation?
A system that looks at itself creates a “point of view.” When the brain models the world and also itself inside that world, it automatically creates a kind of “center of perspective.” That center is what we call a subject. That's what happens when a system treats some information as belonging to the system. How the border of the system is drawn differs, body, brain, mind differs, but the reference will always be a form of “mine.”
The brain can’t see how its own processes work (unless you are an advanced meditator maybe).
So when a signal passes through that self-model, the system can’t break it down; it just receives a simplified or compressed state. That opaque state is what the system calls “what it feels like.”
Why isn’t this just a zombie misrepresenting itself? The distinction between “representation of feeling” and “actual feeling” is a dualist mistake. The rainbow is there even if it is not a material arc. To represent something as a felt, intrinsic state just is to have the feeling.
I argue that the inference bottleneck of the brain leads to two separate effects:
While both effects result from the bottleneck, the way they result from compression of different data streams should show different strength for different interventions. And indeed that is what we observe:
From DeJong et al. (2019):
https://scispace.com/papers/alcohol-use-in-pregnancy-1tikfl3l2g (page 3)