I liked this, but I observe that I don't think it would have gotten nearly as much karma if it had been written by someone without a reputation already.
...which suggests that we're maybe doing something wrong?
Or is this in fact more informative, because we know that it came from Eliezer?
What the system is missing is the "ratio between reads to upvotes". When you just look at upvotes, that's a retrospective quality signal conflated with the prospective calculation people do about whether to try reading. The latter and earlier decision is obviously going to be username-dependent.
Yeah, I've experimented with showing that piece of information, but sadly it made people feel super self-conscious about even clicking on any post, because it would kind of be an implicit downvote. It's also kind of hard to determine what counts as a view (does any click count? Or any time someone reads to the end? Or sometime someone spends more than 10 seconds on the page?).
Plausible we should experiment more with it. Or maybe do kind of a thing where you can get that piece of information but it's slightly inconvenient (maybe putting it on-hover is good enough).
It feels like an important metric is "karma per view" or "karma per read" or "karma per user-minute-looking-at-text" something similar. Currently, we can't gauge that, and so when someone who gives a strong prior that their post will be worth reading posts, that post will get more views which means more upvotes even if they have a similar "karma per read".
EDIT: EY's post loaded the second I posted this, but I promise it was an independent invention
And ironically, your post has substantially less karma than Eliezer's does, despite saying basically the same thing!
I note that when I ran the Conor Moreton experiment, it seemed to me to confirm that karma and attention does accrue to good writing without prior reputation. Perhaps it takes a couple of rounds, to get past people's ordinary why-should-this-be-promoted-to-my-attention-above-everything-else-competing-for-that-attention heuristics, but.
Nobody had a clue who Conor Moreton was, and after a week, he was being listened to.
It definitely suggests karma is an imperfect measure of quality/usefulness but I don't think that automatically means we're making the wrong trade-offs.
(these thoughts are more general than this particular post, which I think is a little different just for being fiction).
Some posts require more energy than others to get the value out of them. When you're first looking at a post, it can be hard to estimate what the total return on energy is. In particular "deeply challenging and thought provoking" can look a lot "total bullshit", especially if the writer hasn't put a ton of skill and effort into making their challenging post easier to access. When faced with a new post that is definitely going to take a lot of energy and has an uncertain payoff, considering the reputation of the writer can be a legit useful heuristic.
This has a bunch of negative consequences, but it's really unclear to me if those outweigh the benefits, or if something could be done to improve the pareto frontier.
I often wonder what effect the idea of Santa Claus has on children's cognitive development. We have created a vast conspiracy to deceive young children, spanning every possible layer of authority: your parents are in on it, your teachers are in on it, mass media and corporations are in on it, even the government is in on it. All working together to support the idea that a man from the North Pole is going to fly around the world and bring you presents.
As a child gets older, maybe pieces of the story start to not make sense. The child expresses a little doubt, but is quickly reassured. The child may think, what's more likely: that my doubts are misplaced, or that everyone in the world I've been taught to trust is wrong or lying to me?
Then finally the child's ability to suspend reason is pushed to its breaking point. Yes, everyone else is really is wrong. My own comprehension of the world is a more reliable source of truth than what I've been instructed to believe. The child finally confronts their parents. "You're right," the child is told. "You've passed the test. You're on the inside now. Don't tell your little sister."
The whole Santa Claus business is either a good idea or a bad idea. If it's a good idea, it could be taken further and further until it starts being a bad idea to go any farther than that!
Hmm... Maybe there should be a way to spoiler-hide tags on posts? Cause not tagging it dath ilan would mean it wouldn't be discoverable through the tag page, which also isn't great.
I didn't look at the tags before reading. I did notice it was fiction pretty quickly but "is this dath ilan" was still a live question for me until the reveal. (Though Eliezer might want to continue writing some non-dath ilan fiction occasionally, if he wants that to continue to be a likely thought process.)
If there is actually such a thing as Security, the best thing to do is whatever you think will keep you (or other people) alive. Mouthing off to a security officer about your actual reason for saving the redhead is behavior that is suboptimal because it would lead to your death. So testing kids to "see if they'll reject conformity" is actually testing kids to see if they'll pointlessly risk their life.
Furthermore, the belief that every sapient being is precious is something that the child was told. In other words, the child cannot claim "well, nobody reasonably argued for the exception" because nobody argued for the rule either!
And I don't see why the child doesn't say "you lied to me about Security. How do I know you're not lying to me about (insert every random thing about the society that the kid might find slightly questionable)." For that matter, the kid could think that "there's no security" is itself a lie and that the whole thing is a Hundred Flowers Campaign--the actual purpose is not to test for compassion, it's to detect troublemakers by giving them a few weeks to fully reveal themselves. It's your world, which isn't actually a Hundred Flowers Campaign, but how does the kid know, now that he's been lied to at least once?
Quite a good story. But I think at this point I would quite like Eliezer to make some sort of statement about to what degree he endorses Dath Ilan, ethically speaking. As a fictional setting it's a great machine for fleshing out thought experiments, of course, but it seems downright dystopian in many ways.
(I mean, the fact that they're cryopreserving everyone and have AGI under control means they're morally "preferable" to Earth, but that's sort of a cheat. For example, you could design an alt. history where the world is ruled by a victorious Third Reich, but where Hitler got super into cryopreservation in his old age, and poured a lot of resources and authority into getting the populations under his control to accept it too. Probably in the long run that world is "preferable" to the world where the Allies win but billions more brains rot — but it's still not much of a utopia in any useful sense.)
Of course, Eliezer previously stuck the non-consensual-sex thing in Three Worlds Collide, as an attempt to simulate the “future societies will likely trivialize things we consider unthinkable and there's no way to tell what” effect. I suspect — hope? — some of the ickier parts ...
The child does not even ask after the red headed kids? If the red-headed kids are real, sentient beings, this experiment is going to damage near everyone involved; we know the social problems and trauma caused by the Stanford prison experiment, and that was consensual, informed and contained.
The experiment also fundamentally underestimates what it takes to resist indoctrination. If your government systematically oppresses a group, and wants you to go along to it, they will not teach you universal rights spanning across these groups and so set you up to be for the minority and against the government. The nazis certainly did not teach such things; rights were for "true Germans", with the strong implication that you needed a genetic claim and to prove yourself in ways only vaguely specified and that it would be very unwise to push. The nazi government also took great care to give you comprehensive reasons for the hate, to build a whole propaganda complex to make sense of why you should hate the jews. Resisting that is a lot harder than going "you said all sapients have rights, Jews are sapient, so what gives". Political systems that systematically abuse subgroups tend to be very clear...
Predictions over the course of my reading:
Guessed "dath ilan" from title.
Switched prediction to "Amenta" at mention of red hair.
Switched back to "dath ilan" (with much higher certainty) at the first use of capital-C "Civilization".
What were the lives of the red-headed children like? Were they in the know about the nature of the experiment? I suppose they must have been, but then how could such small kids be such good actors? I'm doubtful about the power of stories versus someone's experience, but I liked the story :)
It's explicitly called out that the red-headed children "tended bigger than the rest of us, with more adult facial structures, to the point where you could've maybe mistaken them for very small adults in disguise". I suspect EY's intention is that they are in fact small adults in disguise.
I said, at the end, was that I'd better be getting paid for this, and they all laughed and said of course I was, lots of money, at least as much as my parents were getting, because children are sapient beings too.
This seems like a rather hypocritical thing to say, unless dath ilan had some clever idea for how to implement this compensation that I'm failing to see right now.
If I was a subject in this experiment, there would be no amount of money you could pay me to retroactively agree that this was a fair deal. There's just nothing money can buy that would be worth the years of deception and the hours of mortal terror.
If it was earth it'd be different, because earth has absolutely dire problems that can be solved by money, and given enough millions, that'd take precedence over my own mental wellbeing. But absent such moral obligations, it's just not worth it for me.
So do parents surreptitiously ask their children what sum of money they'd demand as compensation for participating in a wide variety of hypothetical experiments, some real, some fake, years before they move to a town like this? Seems rather impractical and questionable, considering how young the children would be when they made their choice.
All dath ilani kids grow up with enough adult gaslighting that they don't just grow up to believe anything authority tells them.
This was just the brand on offer in a particular village.
I am fairly confident that I would incredibly strongly dislike being lied to like this even if it were „for my own benefit“. The source of my disgust for nonconsensual lies does not seem to me to stem from a history of such lies hurting me. Rather, they just feel inherently hurtful. That's on top of the distress I would feel from years of keeping my mouth shut and my head down regarding the fake discrimination while secretly crying about it sometimes.
Also, there‘s still the hours of mortal terror that this scenario entails.
Given your perspective, you may enjoy: Lies Told To Children: Pinocchio, Which I found posted here.
Personally I think I'd be fine with the bargain, but having read that alternative continuation, I think I better understand how you feel.
A thing that feels a bit confused in this discussion is... currently the default state is that adults gaslighting kids... just happens all the time? One possible world you could try to engineer is a world where this never happens. Another world is one where it happens deliberately in a controlled fashion that teaches valuable life lessons and leaves children more resilient. The question is whether the former is actually tractable.
Presumably, from behind the Rawlsian veil, you might accept this deal in exchange for avoiding dystopia. I have not read the Dath Ilan glowfic, but I infer they are genetically/memetically engineered to me much more psychologically robust than us.
Also note, we do much worse to children all the time.
But yeah, it's sort of icky to live in a town where all the adults are conspiring to lie to you, but the vast majority of children in this world are not being lied to. And I think they made a lot of money, presumably the amount of money this rather-competent society predicted would be their "cheerful price".
I would certainly accept such treatment for 2 million dollars, for example.
Though, this fellow seems to enjoy having this story to tell - so perhaps his cheerful price was rather low!
There's just nothing money can buy that would be worth the years of deception and the hours of mortal terror.
Ridiculous. I would do this at least a dozen times for a billion dollars. This is practically the default society in human history. You would get over it.
I think I can model my own preferences better than you can, thank you very much. Regardless of whether I‘d „get over it“ or not, this experience would bother me more than anything extraordinary I can think of that I could plausibly buy in dath ilan‘s economy would please me.
I say this because I think it relates to / could be within your same world:
One of my favorite current story ideas is to write a story where there's a girl who lives in a world where each society is running an experiment about the best way to live life. In school, she finds out that while most societies are aware of the experiment that's being run, her society is one of the few where the experiment is kept secret from the people in it.
As she grows up, she realizes how poor her society is, and how little technology they have. It improves very quickly over the years, but when she was younger they didn't even have washing machines or dishwashers.
So, she starts digging to try to figure out why it is that all their technology was so bad, and why her society started off so poor, and begins advocating for the idea that whatever experiment is being run, it should be shut down, because their quality of life is so much lesser than other societies.
Eventually, the movement she starts begins gaining momentum, and the entire society begins to revolt against being in the experiment. When she's finally able to bring her case to the main governing decision body (the council), she finds out what the ...
I don't understand one thing about the universe. The test and the fact that this story wind "lies-told-to-children" storytelling night, seems to indicate that most children never hear those dirty-red lies. However, if that causes those children to be raised, to learn, differently than most of the planet, this only mean that the test checks whether those children are actually good, not whether the whole planet education system raises good children. This is especially important as I can imagine and fear that seeing this discrimination for years, then having ...
I've a regret that the requirement to write down thoughts only came once. Clearly, there was no way I ever could see where it goes at the time it was requested. However, later, as soon as Red was mentioned, I guessed the end, the tests. Still cried reading it. I suspect that if I didn't know who the author was, I would not have guessed, that seems to unusual for most author authors I love
(I was misled initially because I wound up thinking more Star Trek redshirts or possibly behavioral genetics than red teaming; out-of-universe reasoning can be tricky.)
Related story: https://slatestarcodex.com/2014/06/05/dont-be-an-asch-hole/
It is in fact an Amenta reference (Alicorn/glowfic setting in which red-haired people are discriminated against).
There wasn't actually any such thing as Security, and if there ever was it would mean that it was time to overthrow the government immediately.
I held back tears at this part.
I think that the idea of dath ilan being better at solving racism than earth social media is really valuable (in basically every different way that dath ilan stories are valuable, which is a wide variety of extremely different reasons). It should be covered again, at projectlawful at least, but this is a huge deal, writing more of it can achieve a wide variety of goals, and it definitely isn't something we should sleep on or let die here.
I feel like Conservative Judaism does this pretty well.
In Reform Judaism, everyone is in on the joke. Which means there's no obstacle to overcome, nothing to learn. Reform Jews win no points on Lies-Told-To-Children storytelling night.
Orthodox Jews play the game too seriously. Like, you get a bunch of people who win the game, and they tend to be interesting people who honed an important skill, but far too many don't get it during their lifetimes. Which means that they have these awful ideas like "we are God's chosen people" and "the world exists so we can ...
Curated.
>! At first, I thought about this piece as a comment on educational systems, but that's not really it. The post is about how a society attempts to audit itself and its education. Our community is not at the scale where this setyp is feasible (nor do we necessarily have the time), but it does get me thinking about what are reasonable tests that demonstrate whether we're morally and epistemologically on track.
Growing up, as a kid, I was always told that every sapient life is precious, everything that thinks and knows itself -
Between this beginning and the title, I thought this was going someplace completely different.
Excellent short story!
Curious if anyone has any non-plot-related takeaways that might be enlightening.
When you realize that they totally have a city somewhere that's running the control trial.
This is the first thing I've read set in dath ilan and I think I'd enjoy reading more.
I see the tag wiki lists several other writings in dath ilan, would anyone here care to suggest a reading order for me?
Was this an attempt to flesh out why Dath Ilani would have low posteriors on there being a conspiracy?
Hmm. Yeah, it sure looks rigged as hell to be resolved by self consistency/reflection to the side of "care about everyone", but surely there is some percentage of kids who come out of this with reflectively stable redhead hatred? Or, I don't know, "nobody deserves care, not just reds, but you should pretend to care in polite society"?
So i read a lot of dah-ilani glowfics the previous weeks, and yet, i didn't guess right. i didn't stop to put numbers on it, so i can in retrospective (and still not sure it's actually good idea to put numbers on all things). and it was 0.9 that the stroy is about kid losing trust in adults because they was told a lie, and 0.3 that after that, it turned out they should trust adults and this distrust is bad (like teens that think all drugs are not dangerous because adults exaggerate the harm of the less-harmful ones). in that situation, i was basically 50-5...
Really engaging! Was especially gripped at the bluffing section, and laughed out loud in shock at the reveal. Really enjoyed it :)
I think it would be more accurate to say that the test was meant to check whether the TV shows were effective than whether the children had a maximal inherent tendency towards virtuousness.
Growing up, as a kid, I was always told that every sapient life is precious, everything that thinks and knows itself -
Yes, this is a tale about lies-told-to-children. You'll probably figure it out yourself before too long. For now, just listen.
Where was I? Right. As children, we were always told that every sapient life is precious. It was told to us by the teachers, and shown to us in children's television - though I saw less children's television than most children in our age cohort - children's TV was censored where I grew up, though, of course, I didn't find that out until much later -
I see you're starting to guess under what sort of circumstances I grew up. Go ahead, write down the prediction if you want. Maybe you already see where this entire thing is headed. But you asked me for a story about the lies I was told as a child, and that's what you're getting. It's not my fault, if a lot of stories like that are predictable; people who lie to children have other things to optimize for than unpredictability.
So where was I? Right. I grew up in a remote village of about three thousand people, the sort that's more hills than houses. Charming travel-pathways that cut through forests. Not everyone knows everyone, but you sure know somebody who knows anybody.
Children's television in my region was censored, though of course they didn't tell us that as children. But the children's television that we saw had aliens and monsters and creatures of fantasy, with four legs or fourteen legs, three faces or no face at all, and all of them were treated by the television show as having lives that meant something. Sometimes in the children's show there were alien monsters who only thought their own kind of life was valuable, and then maybe you couldn't trade with them as friends. Maybe they'd already lied to you once and you couldn't trust them enough to bargain with them, maybe you couldn't talk to them at all. But their lives still had meaning to the story's human protagonists, even some aliens whose lives had no meaning to themselves. You didn't cause them pain if there was any way to avoid it; you didn't kill them unless their biology was sufficiently similar to human that you were confident in your ability to cryopreserve them afterwards.
The shows never spelled it out, never said, 'And this is because of a universal rule in every case that sapient life has value.' Our teachers said that explicitly, though.
And they treated every one of us children, too, as if our lives had meaning.
Except the children with the red hair; those dirty reds.
You're nodding along with a knowing look, I see. Was it what you predicted? Not exactly, maybe, but rough ballpark? I suppose I'll find out when we open your prediction afterwards.
The red-haired children hardly needed the red hair, as their targeting-mark; they looked different from the rest of us in other ways too. When I was old enough to first ask, I was told that they were the children's children of people who'd been exiled from a faraway city for committing terrible crimes there, who'd been given sanctuary by the grace and mercy of our own benevolent kind. The red-haired children tended bigger than the rest of us, with more adult facial structures, to the point where you could've maybe mistaken them for very small adults in disguise. The red-haired adults, what few of them we ever saw, were correspondingly huge and muscular. You could see, in retrospect - if you were actually trying to think at all, which we weren't really - how somebody might have felt threatened by such big muscular people, even while graciously granting them sanctuary.
There weren't many of the red-haired children being educated alongside us; a handful, four or six. I can't recall how many by counting names, because they kept to themselves and did not try to be friends with the rest of us.
They were slower than the rest of us in class to answer. On the rare occasions a teacher called on them, they'd often get the question wrong. We were kids, young kids, so of course we didn't ask ourselves anything like "Is this in fact an intrinsic deficit of intelligence or is it a self-fulfilling prophecy about who gets more effort from the teachers?" or come up with any experiments to test that one way or another. We just wordlessly thought that red-haired kids were stupider; and that this too was a universal rule just like gravity.
We did not, in fact, treat our red-headed fellow kids all that well. We were of an age where kids take their cues from adults without carefully rethinking everything they're seeing. We noticed how the older kids treated red-haired kids, we noticed how teachers treated red-haired kids, we noticed the huge red-headed adults who were silently sweeping the hallways and not doing any intellectual labor. We noticed how the adult reds got casually shoved aside by other adults or even non-red-headed older kids, and how the red-headed adults just silently took that.
There were names to call them, 'dirty reds', worse things than that, scatological profanities to giggle over amongst ourselves.
Now and then you'd see a Security Officer come by and ask some reds some questions. One time Security took one of the janitors away, and then after that, nobody ever saw him again. I think one of the kids did ask, in class, what happened to that guy, and the teacher shut her right down and said that any questions about dirty reds or for that matter Security were things best asked in private if you asked at all.
And meanwhile the television shows, those that we got to watch, went right on teaching the lesson that all sapient life is precious, with no exceptions for fourteen legs or not having a face.
Eventually, of course, it started coming to a point, and then it did come to a point.
It started to come to a point, at the point where a red-haired kid was called on in class and answered a question wrong, and the teacher asked if his parents were too busy stealing other people's books to teach him how to read. The red-haired kid didn't say anything back, but I flinched, visibly.
It came to the point, two days after that, when I was walking home from class, and I heard a groan from off the pathway home, what sounded like a moan of pain.
I left the pathway and ran around a hill to find one of those dirty reds hiding behind it, with blood all over his left pants-leg.
He asked me not to get an adult.
He said that he was hiding from Security.
He asked me to help him walk, help him get away.
It didn't feel real. It felt like I was inside one of the children's television shows.
Of course, in children's television shows, they always show the heroes reminding themselves that things are real and that they've got to do what's right, because it's real, so I knew that I needed to remember that this was real because that's what you do when you're inside a television show.
I think I was probably very scared, though I don't remember noticing myself being scared.
I asked him what he'd done to get Security looking for him.
He said that he had, a few days ago, said something about red-haired people deserving better treatment than they currently got, around a non-red-haired person he'd thought, hoped, was a friend.
I gave him a hand so he could stand up, on the leg that wasn't covered with blood, and then he leaned on me and we hopped away through the hills until we got to where a red-haired woman - you saw fewer of those - whispered a thank-you to me and took him away with herself.
I ran back to the pathway and ran home, though I was still late, of course. My dad asked me where I'd been and I said I'd seen a funny-looking butterfly and run off to chase it. I remember believing, even then, that he knew I was lying, but dad didn't ask me any more questions, and I didn't tell him anything.
About an hour later, Security knocked on our door and asked everyone if they'd seen a red-haired person who looked like - and of course the picture was of the man I'd helped to get away.
I said no, I hadn't seen him. But because I was a kid and kids that age aren't taught theory-of-deception, I asked what the man had done and if he was considered dangerous. And I didn't think, until too late, about whether that was something I was much more likely to ask if -
The Security officer asked me if I maybe wanted to change my mind about having seen the fugitive.
I gave him my best surprised look and said no.
The Security officer noted that Security officers get special training in reading emotions, and I seemed pretty frightened to him.
I said yes, I was, because the Security officer was suggesting that I was lying and that was scary.
The Security officer said he knew perfectly well, at this point, that I was lying. But I wouldn't end up in trouble if I showed him where the fugitive went and identified anyone else he was with.
I said that he didn't know what he was talking about.
The Security officer gave me a sort of stern look and said that he'd detected another lie, and did I really want to get in trouble for some dirty red.
I told him that I wasn't stupid and I knew he was bluffing, to try to trick me, because he suspected me, even though I hadn't done it.
He took a photo out of his pocket and showed it to me.
It was me helping the red-haired man walk on his one good leg.
Why, said the Security. He just looked sad, now. Why had I done it? Why was a dirty red worth it?
And I remember, by that point, that I'd noticed I was scared, and I think I was trying to get out of it - by proving that I was, in the end, obeying adult authority - when I said that we'd all been told in class that every sapient life is precious, everything that thinks and knows itself, that was the rule we'd been given, and nobody had reasonably argued at any point that there was an exception for people with red hair, and also we'd all been told that hurting people is wrong and you shouldn't let social conformity push you into it.
The rest of it went the way you'd expect.
The Security officer smiled.
My parents rushed in and hugged me and told me I'd been so brave and so good and scored in what would've been the upper 5th percentile twenty years ago for the age where I started to object and not go along with it anymore; and explained about Civilization needing to test some kids now and then, to find out how well we were doing environment-wise and heredity-wise on people's kindness and resistance to conformity-pushed cruelty; and test against an earlier-reported bug where general rules about fair and okay treatment of people would somehow end up not being applied to some subgroup; and our little village was settling an important conditional prediction market from twenty years earlier, that had millions of labor-hours wagered on it; and that children growing up to be good people was a vital figure-of-merit for all of Civilization and lots of big policy decisions turned around it, which was why it had been worth specializing our village to do Science about that, and they hoped I understood all that and wouldn't tell the other children right away. There wasn't actually any such thing as Security, and if there ever was it would mean that it was time to overthrow the government immediately.
I nodded along in a wise, understanding, and rather numb fashion. I think the main thing I said, at the end, was that I'd better be getting paid for this, and they all laughed and said of course I was, lots of money, at least as much as my parents were getting, because children are sapient beings too.
So that's my story about the-lies-we-tell-to-children. And the part that I value now the most, even more than the money I got then and when I was older, even more than knowing that I was good and brave in the only sort of real test that most people in Civilization ever get, is that I approximately always win any Lies-Told-To-Children storytelling night.