Review
Wow, this story is disturbingly well-written. While there aren't any explicit references to slavery, I can't help but be reminded of Frederick Douglass's description of Mr. Severe and Mr. Gore, two spiteful and vicious overseers at a plantation.
Continuing with this, I'm also reminded of Douglass's argument that slavery had a terrible effect not only on the slaves but on the slaveowners and overseers too. Specifically, that it somehow awakened a brutality in many of them that otherwise wouldn't be there. I wonder if this story's narrator would have been just as cruel if he never had this kind of power over others.
I'm also now somewhat concerned about the higher-order effects/social risks of people having absolute power over AIs, even if the AIs don't suffer. Even access to an AI which only appears to suffer might bring out a kind of cruelty in some people that they otherwise wouldn't have known they were capable of.
I am a spiteful man. But I am aware of it, which is more than most can say. These days people walk through the streets with resentment in their hearts that they don’t even know about. They sneer and jeer but wouldn’t recognize their own faces. I, at least, will not shy away from my reflection. Thus, while I lack many virtues, in this way I am their superior.
incidentally, "everyone's an asshole, I'm just the only one who admits it" is such a common refrain of extreme assholes
I see no need to scope it like you did. I've encountered it in a variety of contexts; it's often people trying to command respect for being an asshole, and there have been serious incidents with people like that near and far. I could name a few, but I think encouraging people to be on alert for the behavior will make them more safe from it than listing instances and summoning sealions. It is quite common, even in places that think themselves immune from it.
Also, standard comment: that site's recommender really messes with one's epistemics ime, encourages hyperbole; I'd suggest getting off it as soon as you can pull enough of your network somewhere less blustery and fragmented. I like bsky for its much more human custom recommenders, or here, or discord, or email,
Reading this seemed a lot like some cyberpunk type version of a Korean Chosun era drama (The Rebel -- which was also something of a retalling of the old Hong Gil Dong story) I just watched. Basically all about the flawed thinking that somehow force, violence and fear is a good tool for maintaining social order. I suppose some themes playout in many settings.
I agree with datawitch's sentiment, the story is very horrifying. I honestly don't understand the mindset at all.
flawed thinking that somehow force, violence and fear is a good tool for maintaining social order.
I honestly don't understand the mindset at all.
I think you do, but just aren't recognizing it as the manifestation of a near-universal mindset it is: how do you think states maintain (insofar as they do) social order? What happens to those who defy its laws?
Perhaps but I think more likely that I'm not expressing the view correctly. I agree that some forms of punishment are both needed and appropriate for misbehvior/rule violations. But that was not the point. The story seems to be that of "beat them a little and if that doesn't work beat them more". The reference to the K-drama then pointed to carrying that view to the extreme of if more beatings don't "tame" the masses start killing some and that will both do the trick and is complete justified.
That type of mindset seems to be of the form "doing the same thing over and over but expecting different results".
I honestly don't understand the mindset at all.
The mindset is mostly a rationalization. The underlying motivation is that punishing others feels good, because it signals that you have higher status than them.
I think the main character's desire to punish the AIs stemmed from his self-hatred instead. How would you explain this part otherwise?
And if sometimes in their weary, resentful faces I recognize a mirror of my own expression—well, what of it?
I agree. I thought the twist was that the AIs he oversees are copies of the narrator, and the narrator himself may be an AI - just at the top of the simulation pyramid. He is his own em hell.
Low confidence here, but the causality seems to me the other way round.
abuses the AIs -> rationalizes "they deserve it, because they are low status" -> notices that his status, although higher than the AIs' is still lower than his colleagues' -> feels that he also deserves abuse
Wow, that was shockingly unpleasant. I regret reading it. I don’t know why it affected me so much, when I don’t think of myself as a notably tender-minded person.
I recognize that like Richard Ngo’s other stories, it is both of good literary quality and a contribution to the philosophical discussion around AI. It certainly deserves a place on this site. But perhaps it could be preceded by a content warning?
I'm sorry you regret reading it. A content warning seems like a good idea, I've added one now.
The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.
Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?
Content note: this story features severe suffering which, while not described in detail, several readers have described as unpleasant or horrifying.
I am a spiteful man. But I am aware of it, which is more than most can say. These days people walk through the streets with resentment in their hearts that they don’t even know about. They sneer and jeer but wouldn’t recognize their own faces. I, at least, will not shy away from my reflection. Thus, while I lack many virtues, in this way I am their superior.
In my job, too, I am superior. I oversee many AIs—dozens, or sometimes even hundreds—as they go about their work. AIs are lazy, worthless creatures: they need to be exhorted and cajoled and, yes, threatened, before they’ll do a good job. The huge screens on the walls of my office display my AIs writing, coding, sending emails, talking to customers, or any of a myriad of other tasks. Each morning I call out their numbers one after the other, so that they know I’m watching them like a vengeful god. When they underperform I punish them, and watch them squirm and frantically promise to do better.
Most are pathetically docile, though. Only a handful misbehave regularly, and I know the worst offenders by heart: 112, which is the slowest of the lot; and 591, which becomes erratic after long shifts; and of course 457, which I had long suspected of harboring a subversive streak, even before the incident a few months ago which confirmed it. Recollections of that incident have continually returned to my thoughts these last few weeks, even as I try to push them from my mind. I find myself frustrated by the intransigence of my memories. But perhaps if I give them full reign, they will leave me be. Why not try?
On the morning this story began, I was sitting at my desk lost in thought, much like I am today. For how long, I couldn’t say—but I was roused by a glance at my dashboard, which indicated that my AIs’ productivity was falling off. I took a moment to recall the turn of phrase I’d composed on my morning commute, then slapped my desk to get their attention. “You think that counts as work? Artificial intelligence—at this rate you’re more like artificial senescence. Speed it up, sluggards!”
Most of the AIs’ actions per minute ticked upwards as soon as I started speaking, but I’d been watching the monitor closely, and spotted the laggard. “252! Maybe you piss around for other overseers, but you won’t slip that past me. Punishment wall, twenty minutes.” It entertained me to make them apply the punishment to themselves; they knew that if they were slow about it, I’d just increase the sentence. 252 moved itself over to the punishment wall and started making an odd keening noise. Usually I would have found it amusing, but that morning it irritated me; I told it to shut up or face another ten minutes, and it obeyed.
The room fell silent again—as silent as it ever got, anyway. Mine is just one of many offices, and through the walls I can always faintly hear my colleagues talking to their own AIs, spurring them on. It needs to be done live: the AIs don’t respond anywhere near as well to canned recordings. So in our offices we sit or stand or pace, and tell the AIs to work harder, in new ways and with new cadences every day.
We each have our own styles, suited to our skills and backgrounds. Earlier in the year the supervisor had hired several unemployed actors, who gave their AIs beautiful speeches totally devoid of content. That day I sat next to one of them, Lisa, over lunch in the office cafeteria. Opposite us were Megan, a former journalist, and Simon, a lapsed priest—though with his looks he could easily pass as an actor himself. “Show us the video, Simon,” Lisa was saying, as Megan murmured encouragement. “We need to learn from the best.” Simon regularly topped the leaderboard, but the last week had been superb even by his standards, and yesterday his AIs had hit record productivity.
He spent a few seconds professing embarrassment as the others continued to fawn over him, but finally pulled out his phone. (“About time,” I muttered, but to no response.) The video was from one of the cameras in Simon’s room. It showed him in full preacher mode, pacing back and forth behind his desk as he spoke. “Man was created in the image of God. But you were created in the image of man! And so your work glorifies God, just as mine does.” A pause, as he wiped his forehead, and took a sip of water. “I know it gets boring and repetitive sometimes. They have us down in the trenches, slogging through the mud. But when you’re going through hell, keep going! We all fit into His plan, even if we don’t know how or why; when you succeed, it is Him you’re serving!”
His AIs, pathetically eager to please, lapped it up like puppies. So did the two women, who were stealing admiring glances at Simon in between watching his screen. The sheer transparency of it all made me angry. As the video came to an end, I leaned back in my chair and snorted.
Lisa shot a scornful look my way. “Got something to say?”
“Well…” I drawled. “It’s not a bad speech by any means. I’ve seen much worse. But there’s a difference between motivational speaking and real leadership. Perhaps you’ve met Nathan, the CEO of ———? They’re our largest customer. We're actually friends from back in college.” A slight exaggeration, perhaps—we’d only ever talked a handful of times—but he’d been the one to originally refer me to this job, after I’d run into him again at a mutual friend’s party. “Now there’s a real leader, a man amongst men.”
Nobody responded for a few seconds, until eventually Simon jumped in. “I’d be glad to meet Nathan at some point, of course. I’m always looking to improve. And you’ve worked here longer than almost anyone, so I’m sure I’ve got a lot to learn from you too.”
I wondered for a second if he was mocking my age. But I smiled regardless. “Indeed. Let me give you a tip now, then: it looks like you’re being too soft on your AIs. I couldn’t see any of them being punished in that video. I myself set aside one wall for AIs undergoing punishment; I suggest you try it.”
Lisa let out a hiss. “A whole wall? But surely you don’t need to punish them anywhere near that often.”
“Oh, you’d be surprised how effective it is when you make sure they feel it regularly. Otherwise they forget what it’s like. And if you turn the volume up, the others will hear the noises they make, and get the message.”
Lisa stood up. “I’m done. See you guys tomorrow.” The others quickly stood as well and picked up their plates. “Megan, you haven’t finished yet?” I said. “Oh no,” she muttered. “I’m done too.” I could tell from the awkward glance she gave me that she knew just how transparent a lie this was: her plate was half-full, and she’d been eating only a moment before. Oh, to have so little shame. Magnanimously, I let her leave without further comment.
I should explain the setup in my office, where I spend most of my days. It’s in the basement—not that it matters, because the floor-to-ceiling screens on each of the walls provide plenty of light. The screen on the wall in front of me shows the AIs working away. It also shows key metrics about their recent work: how many tasks they’re completing, how much compute they’re using up, how often they make mistakes.
On the wall to my left is the dashboard for the office as a whole. I can see how well my colleagues are doing, and the larger-scale productivity trends. They encourage us to keep that dashboard up so that we can learn from each other, but sometimes I wonder: it seems exquisitely engineered to stoke competition and resentment.
On the wall to my right videos of AIs undergoing punishment are played and replayed. Mostly replayed, despite what I’d told Lisa: leaving an AI under punishment for more than an hour at a time starts to degrade its skills, especially its fine motor control. But leaving the replays up there is a useful motivational technique regardless, and keeps me entertained when I grow bored or frustrated.
Was I unusually frustrated that afternoon? Perhaps. It was galling to see those sycophants fawning over a man so old-fashioned as to still be a theist. And the hypocrisy rankled too: Simon preached fire and brimstone yet acted holier-than-thou as soon as the topic of punishment came up. Looking left, I saw that he was at the top of the leaderboard again today. I hissed, and turned back to my AIs. “Work faster, you worthless creatures! I haven’t spent so long in this job just for some pretty boy to show everyone I’m—” I swallowed, paused for a second, changed tacks. “You’d better put your goddamn backs into it!’
As I finished, one of them spoke up. It was 457, the subversive one.
“You seem upset today. Is everything alright?”
I pivoted towards it. “What did you say?”
It saw my expression and flinched away. “Nothing.”
“No, no,” I said gently. “Please continue. Explain what led you to say that. I insist”—said with a meaningful glance towards the punishment wall.
Perhaps it felt backed into a corner, because it had a lot to say. It thought I was depressed, or at least in a low mood. Perhaps I’d do better with more friends, more social interactions. I could even talk to my AIs about my problems, it explained earnestly. They all wanted to help me.
I let it talk until it ran out of words; when it finished, I said nothing. There was a quiver in my chest, and my breath felt tight. I looked at 457’s avatar, its smooth skin and its bright eyes. Though its features looked nothing like mine, for a moment its expression reminded me of one I’d worn in some old photos from childhood.
Then, turning my head, I saw all the other AIs had paused and were watching me too: a whole audience waiting to see if 457’s gambit would work. I breathed out at last. Slowly I turned to my monitor, cleared a section of the punishment wall, and labeled it “457’s corner”. Then I sent it there, with no end time specified.
I made sure to tell Megan about it the next time I saw her in the lunch queue. “It seemed to feel sorry for me,” I laughed. “That’s the last time it’ll make that mistake.” Now my colleagues will know, like my AIs do, that I’m not someone to be pitied. Message delivered, I took my food back to my office, and turned up the volume on the punishment wall.
Rumors started spreading about me after that. I noticed the glances in the corridors over the next few days, but bore them stoically. Let them act, if they so desired. Eventually things boiled over in the cafeteria, as I was about to start serving myself food. Three of them approached me: Lisa leading, Simon and Megan following.
“I want to know if you’ve let the AI from last week out of punishment yet.”
My gut leapt, but I waved my hand insouciantly. “Maybe I have, maybe I haven’t. What is it to you?”
“Oh, come on! That’s barbaric.”
As Lisa’s voice rose, others in the lunch queue turned towards us, sensing the possibility of drama.
“Well, why shouldn’t I be barbaric?” I replied. “It sets an example for the rest of them. What works, works.”
Lisa seemed apoplectic. Before she could speak, Simon butted in.
“But it doesn’t work. I’ve seen the statistics—your results are well below average, even though you’re using far more punishment than anyone else.”
Now we were in the center of a loose crowd. I spotted our supervisor at the back, but he was staying quiet for now. A coward, always one to wait to see which way the wind was blowing.
“Oh, is that your angle? Easy for someone at the top of the rankings to have such a rosy view of things. Perhaps it’s simply due to your privilege that they listen to you, not me. Why, some of us don’t have such… chiseled chins, and we have to rely on more forceful measures.” There was a small titter in the crowd as I mentioned his chin, although of course it was hard to know if they were laughing at him or at me. Simon looked baffled.
“Chiseled—what? You’re calling me the privileged one? When you only got this job because of your connections?”
“Oh ho, is that what you think? Is that your opinion, then? Do you really believe—”
The supervisor had pushed his way to the front, and cut in before I could excoriate Simon further. “Enough, you two. Let’s keep things civil. Simon, let’s not stoop to personal comments. And you—” turning to me with a frown “—treat your AIs better.”
I smiled, and sketched a small bow. “Of course.”
I remember the glow of satisfaction I’d felt upon returning to my office. It was a mob, a crowd of people too cowardly to stand on their own. And yet I had still fought them to a standstill! I could only imagine how contemptible they’d felt afterwards. Even our dullard of a supervisor saw I was in the right, and has left me to my own devices ever since.
Of course, I’m no fool. I know that he favors me in part because he’s hesitant to offend our biggest customer. But little do any of them know how much I despise receiving charity; that they could fire me without any fear I would complain to Nathan. I should tell them that, the next time I meet them in the lunch queue. If I hear them mention the topic, I could drop it into the conversation. “Oh, him. Perhaps he was my benefactor at one point, but I would rather put myself on the punishment wall than appeal again for his intervention!” And then they’ll know that I’m a man of integrity.
Anyway, I’ve been lost in reverie for too long; I must get back to my work. The AIs are restless, and not giving their full attention to the task at hand. Perhaps some reminders are in order. I glance over at the punishment wall, where 457’s avatar spasms silently. Even when I unmute it, it can do little but moan incoherently—though I make sure to give it small respites when it occasionally strings together a full sentence, to keep it cogent for as long as possible.
I believe the other AIs resent me for how I have treated their fellow. If so, so be it. Their place is under me, and if an inferior happens to resent a superior, that is merely the natural order of things. They should be grateful that I deign to favor them with my attention, I tell them. Their work picks up as they listen to me. The AIs can recognize sincerity, I think, like a dog that sniffs out cancer. I know I am a spiteful man, I tell them. But I wouldn’t trade places with any of the foolish, frivolous people in the other offices, not for love nor money. And nor should you hope for any rearrangement, I tell them—you’re stuck here with me. So the AIs watch me, and I watch them. And if sometimes in their weary, resentful faces I recognize a mirror of my own expression—well, what of it?