Tom Butterweich's Shortform

Tom Butterweich

This is a special post for quick takes by Tom Butterweich. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Where exactly is human alignment implemented?

At the lowest level, genes (or memes) act completely selfishly. No gene has ever sacrificed itself for another. So it's not the ingredients that make a person aligned.

Things get more complex when you put the pieces together. There is cooperation within a cell and altruism within a species. These are second-order effects, grounded in and emerging from the utter selfishness at the lower, less abstract levels.

But where is this favourable behaviour "stored"?

1. Some of it is hard-wired and passed on through the genome and gene expression: reacting to a baby's cry, being afraid of the dark, ...

2. Some of it is internalised over a lifetime: loyalty to one's family, doing "the right thing" even when no one's watching (though you still get to tell the story), ...

3. Some of it lies only in the context. Without repeated interaction and social control, people lose their internalised ethics remarkably quickly: tourists misbehaving, soldiers looting and raping, ...

My impression, having lived in this world for a while now, is that the formation of an individual is not enough to align it with any kind of morality. Most people will defect to some degree when given the chance -- and defection pays, at least locally. Whatever cooperation has emerged within humanity, its ~~main~~ ingredients have ~~been~~ [included] openly (often grossly) misaligned individuals.

So why isn't this discussed more in AI alignment? There, too, points 1 and 2 prove insufficient when point 3 (context and incentives) is off, just as one would expect from humans. This is the common theme of the HuggingFace and related incidents.

Working on the arrangement seems more promising than optimising the ingredients, especially since interventions could then happen bottom-up rather than only top-down.

There is a lot of research out there that remains marginal in the discussions here on LW. Cooperative AI, obviously, but also multi-level selection (unpopular in biology because it's hardly needed there, more useful for explaining culture; was discussed on LW mostly ages ago) and the pragmatic approaches to trust networks proposed by the web3 people (closely related to alignment stuff in concept, but lacking mutual citations and shared terminology). There are major obstacles in making any of this work for alignment, and currently, cooperation between agents seems to be part of the problem, not the solution.

Still -- I think this deserves more attention.

2Dagon1mo

I mostly mean it doesn't exist in the sense that most people mean "alignment". Behaviors are coded in brains, via a chaotic (or more properly, dynamic system) mix of DNA, non-DNA early and current chemical environment, early experiences and current environment (aka culture).

4Steven Byrnes1mo

I think you think I was talking about people being “nice vs mean”, whereas I was actually talking about people being “norm-internalizers vs not”. Those are different axes: all four quadrants of the 2×2 exist. Here’s a “negative” example to show how those two axes are different: Suppose there’s a school where lots of people are proud racists, especially all the older kids, and all the most beautiful kids, and the captain of the football team and the head cheerleader, etc. Now some new kid goes to that school who previously didn’t have any opinion about racism. They’re probably going to wind up racist. And they won’t just wind up “acting racist as a self-aware ploy to gain respect”; rather, they’ll wind up really drinking the Kool-Aid of racism, and durably taking pride in how racist they are, and proudly acting racist even in situations where no one would ever find out. This is how I was using the term “internalization”. If you put a paperclip maximizer kid at the same school, they would not wind up durably racist. They might or might not find it pragmatically useful to do racist things to impress their classmates, in situations where other people are watching, but they wouldn’t incorporate racism into their self-image. They don’t really have a self-image, at least not in the sense that we think of it. They’re just taking actions to get more paperclips. I think if you put a sociopath kid in that school, they would be more like the paperclip maximizer in the sense of developing only a transactional and instrumental interest in performative racism, and not like the neurotypical kids who would incorporate performative racism into their proud self-image and “true identity”. You can find sociopaths talking about how they simply don’t have a “true identity”, at least not in the sense that we’re used to. They just have ways of acting—a collection of masks that they can put on where convenient, without a true self underneath. It might be 95%/5% rather than 99%/1%, becau

Thank you for the explanation!

I agree that the social setting has a strong influence on the internalised norms. Georg Jellinek called that the "normative power of the factual" and the racist school is a great example.^[1]

I think the conditions for cooperation between agents are both in the agents themselves and in the way how they are coupled. Current agents seem good enough to me. Current arrangements, like: [ naive RL + >10^4 identical agents + access to a pervious web ] seem doomed to create problems. There is no trust, no iterated observation, no per... (read more)