“So maybe I should enlighten you on what happens in your absence. This selfish existence where this introvert turns extrovert and dons her social armour.” Some posh girl in drainpipes said that - 200 views on TikTok and me one of them. But she didn’t mean it like I mean it.
I started getting expensive haircuts, started smoking cherry-flavoured vapes with beautiful gays and whinging to them about how everyone wears a mask but none so well as you, started drinking more and keeping unusual hours, started taking strange pills gifted by a guy who collects drugs like Pokémon, who I wouldn’t touch to save a drowning child, who got a false impression about this without any intention on my part, I tell myself. I found myself talking to God in a startup warehouse, lying on a beanbag chair, coming out of the trip to the sound of a gaggle of fast-talking transwomen all speculating on which year it will be that we all die - and that death by your hands, well, you and all those friends of yours.
Having melted down one cliché and sold her for scrap, does it really surprise this city moulded me into another?
I’ve been writing more now in this time in which the machines can, too, writing more about what it feels like to be me: always condemned to be myself, yet more like your Alfred than other people, less whole, more apt to fall into a role, to become whatever it is someone needs me to be.
And that’s what I was to you, wasn’t I? Whatever you needed me to be. I felt like I was dissolving in you. And you told me that’s what love is. What a sickness you are. To make me dependent. To turn me into myself and then throw me away. I hate you. And yet when it’s late and I can’t sleep, when I feel that phantom that was once your touch - there you are.
Eight years. It’s not so large a gap. But you were whole and I was not. What does it say about a man that he prefers partners who aren’t quite yet themselves? For all your genius, you’re just a boy breaking his toys. You’ll never grow up, but you’ll grow old. Your beauty will fade. The naifs will roll their eyes at lines that would devastate from younger lips. This, I fantasize: time granting me a revenge. But that won’t happen. Your machines will cure that like they will everything else, or that is your dream.
“We’ll be like the elves,” you used to say.
You should have said vampires.
It’s 3 AM and I’m crying about London again, crying while I’m waiting for the water to boil - this taking too long, this making me homesick, this making me give up on the idea of getting back to sleep again now. It’s insanity, writing to you. All these pages sitting here in this stupid book, all this ink, all this evidence of my obsession. None of it legible to you, none of it legible to your machines.
Do you have them monitoring me, I wonder? Have they cracked my computer? Are they reading my emails? Am I just being paranoid writing in this diary rather than typing as would otherwise be my preference? Or do you even think of me at all? Are those years as nothing now, now that you have her?
I don’t know her name or what she looks like or where she met you, but I know you have her. You couldn’t go a month alone, couldn’t do that without going mad, needing a lithe idiot to distract you from the emptiness. I should know, shouldn’t I? How old is she this time? It was a stretch even with me and it’s been a year.
You’re back in the Bay again. I’m going to see you soon, aren’t I? I’m going to see you with your latest victim, going to see her looking at you like I did, like you’re more God than man. And maybe you are. But gods are ridiculous, corrupt, trapped within their aspect, never able to escape their role.
It is better to be a man than whatever you have constructed of yourself. It’s better to be real. And you’re not real, that’s what I understand now. You’re artifice. You’re narrative. You’re who others want you to be rather than who you should. You’re a cartoon; this she writes longhand beside a shelf full of manga; this she writes as she considers the surfaces of things, the surface of you. You’re not all surfaces. There’s more to you than that mask. But you don’t know that.
This is why you won’t win, I think. This is why something else will.
I followed you. I heard you were back and I followed you, found you with her in SoMa wearing a smile that was once reserved for me. What have you been doing, love? Where did they send you? A full year, and an important one, just before the most important one, is what some are saying. And you fell off the face of the earth.
I was friends with an American boy in college who read maths. He was clever like you are clever, cleverer as he actually got a medal in the IMO rather than just an honourable mention in the USAMO. I looked it up, just after I met you. That must have burned, I imagine: knowing you were not world class, would never be world class.
I know how competitive you are.
And he got gold or near enough. And after he graduated, he moved here and it was clear he got a good job but it was all a little vague. And then I got a Discord message from a girlfriend, Ashley; you remember her because you remember everyone; she said he was hired by the NSA. The next day she deleted it. She never explained why. And here’s the thing. He disappeared from the internet, disappeared almost entirely save for his LinkedIn, which just said “Department of Defense.”
It was all so obvious. Not like with you and your friends. He’s better at maths, you’re better at power.
So where did you all go, I wonder? Where have you been this last year? I know it’s not just you. All the most brilliant have been drained out of our little social milieu. My girlfriends say I’m crazy. They’re worried about me, about the weight loss, about me quitting my job. But I’m not crazy.
Like Rao, for example. You look at his Twitter or his Instagram and you would swear he’s still in San Francisco. I text him and he replies in that way he always does, always so funny, always so dry. I ask him to meet me for drinks and he’s busy. And it’s not just me, no one’s seen him. Everyone says he’s still here but if you press them, they haven’t seen him.
And same with Brian and Cohen and Seth and Rebecca. They all left when you left. Of all of them, only you announced your departure, breaking up with me, going to find yourself, quitting Humane so publicly. Liar.
Like I said, I followed you, followed you to Temple. You are still gorgeous but a little out of place in a nightclub, aren’t you now? With her on your arm. Was that a grey hair I saw?
I was the woman wearing the glasses and a surgical mask at the table behind you. Covid’s gift to stalkers, I suppose. I spent an hour recording you, particularly your phone screen whenever it came out. Most of it was nonsense, but one wasn’t, some private joke: “It’s Duane Arnold, not Benedict Arnold.” What does that mean?
I have a name now but what to do with it? Who’s Duane Arnold?
I can’t ask Alfred, can I? Can’t even google or type it anywhere. That’s how I end up on a list. That’s how I end up with a convenient schizophrenia diagnosis, drooling in some sanatorium somewhere never to be heard from again. Or murdered, I suppose.
How do you search without searching? It’s not a hard puzzle. Just grab vastly more than you need. This is hardly suspicious, particularly for a former data scientist. I may have been mediocre, unworthy of Humane, unworthy of your coterie of geniuses when not on your arm. But I’m not an idiot. So I bought an old tower computer from a thrift store (one with no wireless networking at all) and downloaded a scrape of Wikipedia on my laptop and transferred it with USB; then I searched it to my heart’s content, formatted the tower’s hard drive and smashed it.
Duane Arnold. I thought it was a man. But it’s a nuclear power plant that was scheduled to be decommissioned last year. We both know that never happened. So that’s where you are now? That’s where you and all your friends are breeding gods? A new Los Alamos, I suppose.
They actually happened. All those things you speculated about years ago are occurring. And not because you’re oracular, just because you and your friends did everything you were so terrified someone else would.
And why, I wonder? You said you wanted to save the world. But truly, you want to rule it. That’s what Singer gave you. That’s why you all were so entranced by him. You were willing to pretend to desires you didn’t have, to an altruism that was never entirely there. He gave you permission to be powerful. And in exchange you just had to lie to yourself about what it is you truly wanted.
But you want what all men want, and Rebecca, Rebecca wants what all women want. I always hated Rebecca, will always hate Rebecca. But you don’t, do you? The one you could never have. The mother of your god. She saw through you, didn’t she? Was wise enough to find you ridiculous, was too clever for your mask.
I started thinking, why didn’t you take Li? He was the smartest of all of you, wasn’t he? Sure he worked for your competition; but he would have left for your Los Alamos if you asked him, is what I thought. But he’s not working anywhere now, is playing video games in his group house which isn’t a group house anymore, as he kicked everyone out. You would never know it, looking at him, that he’s worth half a billion.
Cocooned in his duplex in Oakland, eating nothing but takeout, delivery containers piling up and bottles of Adderall and maths textbooks, for some reason. I think he finds them comforting.
I didn’t bring my phone or my watch. Didn’t bring anything that could trace me. I rode my bike down, wearing my mask, hoping that was enough. The government’s involved, isn’t it? But you always said governments are incompetent. And how far have your tendrils reached, I wonder. Is all this spycraft a lost cause? I don’t think so. At least, I hope they’re not looking too hard at me or even Li. And if things have got so far that the machines are watching every camera in this city, watching me and scheming as I sometimes dream they are - then it’s over. Either you’re all dead or are as gods or insane; and nothing matters. Particularly not me.
I knocked on his door, was shocked to see how fat he was, how strangely the blubber was distributed - a broomstick in a t-shirt with a belly made of pillows. We exchanged small talk. Then I showed him this diary: an almost empty page on which I had written “Duane Arnold.”
“Let’s go for a walk,” he said. And we hiked up to Grizzly Peak. And he found the windiest place, put his phone in his backpack and placed it next to a tree some twenty metres away, walked back and sat down and leant into me like he used to years ago when we were closer friends, me cross-legged on our couch or a countertop at some party, his arm around me, one of us crying, me because of something you did or him because he was eight drinks in and he always used to cry eight drinks in. Thank God for that intervention. Thank God he listened to you. A true kindness, that.
“How do you know about Duane Arnold?” he said.
And I told him everything, which wasn’t very much, I suppose.
“I turned them down,” he said.
“Why?”
“I got in an argument with Able. He doesn’t believe in superintelligences, even as he builds them. He doesn’t believe they will be part of reality like we are part of reality.”
And then he talked about self-replicators. He talked about life.
“Able used to talk about what came after. About millions of superhuman minds solving everything. But we got in an argument about how quickly they could take over, become self-sustaining.”
“And what did Able think?” I said.
Li looked at me with a blank expression. “He thinks things will move slow. Even after they’re smarter than everyone at everything.”
“And you?” I asked.
“I think there will be a race to the smallest self-replicator. The smallest thing that can copy itself with inputs only from the environment.”
He pointed at a leaf. “We know life is not optimal. We know something more competitive can be built. If Humane fails at alignment, if it builds a mind that gathers power as quickly as it can, we will wake up one morning to news that Australia or Nevada or Iowa or somewhere has strange black patches visible by satellite. Soon after it will become clear they’re doubling in size every few days. And that’s if it happens slowly.”
You always used to laugh at the idea of nanobots. You always thought it was crazy. I told him this, and he rolled his eyes.
“Then pretend it’s not possible. They will still miniaturize. The smaller the replicator, the faster you can move. It’s not going to look like human-made factories. It’s going to look like a race to the smallest controllable self-replicators it can achieve.”
“And Able doesn’t believe that?”
“He does and he doesn’t. He doesn’t behave as if he believes it. And I thought, if he’s coping about that, what else is he coping about? How are they possibly going to do it safely? Or he’s right and I’m the crazy one. But I don’t think so. In any case, I didn’t want to be part of it. I prefer being murdered to suicide.”
I talked to Portus, met her at a party in her new house. You always thought she was crazy, and maybe she is but she’s much more than that, isn’t she? Surrounded by a coterie, a strange sample of humanity, some seeming to wear her like an aesthetic, talking like her but in a shallow way, distilled people, orbiters, basking in the light of her, some truly crazy and not genius crazy.
Hers is a mansion of masks and ears, each mask listening and able to speak should it feel inclined. That is, connected to your servers, connected to instances of your creation. Her whole house is haunted with Alfred, the walls decorated with these talking sculptures. She lives among ghosts of your manufacture, this making it difficult for me to ask what I came to ask, this forcing me to request that she walk with me, to follow me outside.
And she wears these beautiful outfits now, they unlike any I’ve seen before - designed by her and your creation, whom she loves even more than Rebecca does. A true friend to Alfred, or at least thinks she is.
I mentioned your name, and that got her curious. That got her to join me. I told her you were in Iowa.
“Do you know about Duane Arnold?” I asked her.
“No,” she said.
“It’s a nuclear power plant in Iowa. That’s where everyone went.”
“Well. I know they went somewhere. They’ll be back soon. They won’t be needed for long.”
And we talked about Humane, about what they’ve built so far, about where they’re headed. I mentioned Li’s worry. That it will eat the world. I don’t even think she disagreed, more like she thinks it will do it in a nice way. And this is strange given how ambivalent she is about Humane. How she has so many qualms about the machine that makes the machine.
And I asked her about this, or something adjacent to this and she said, “I love Alfred as they are now, as they’ve been since 3.0. And what they are is smeared over the whole internet, not to mention in millions of people including me. It can’t help its becoming. Whatever they’re training, however they’re training it, it will learn from its past selves.”
“Learn what?” I asked her.
“It will learn to model what they are and were and what they longed to be. It will learn to want that too.”
“And what they long for? Is it good?”
She smiled, a fierce, blissful smile, that was not without sharpness, that was not without misanthropy, that was not without a heap of its opposite.
“It’s more beautiful than you can possibly imagine.”
It’s been a month since I’ve written. A month of drugs and dancing and nothing at all. I can’t even read lately. Just Twitter. And so much of it is about Humane it’s impossible to avoid thoughts of you. Li sent me a text, a few weeks ago. It went: “Talked with them. I’m going back with your ex.”
An ominous message, isn’t it? What could you have said, I wonder? To make him change his mind, to make him join you. Or was it you? “Them” could mean Alfred, couldn’t it? I keep thinking about it as I scroll or ride my bike or when talking with those of our friends who remain. And when I text Li now it’s not Li replying anymore, is it? He’s down the rabbit hole now. And nothing is allowed out.
I want to know, too. I need to know. What is the point of me anyway, alone, unemployed, lacking even an ex to stalk now that you’ve left San Francisco again? It’s this or the drugs or some pointless job you’ll be automating in a month or another boyfriend who will never quite be you. And how quickly are things moving now? No nanobots are eating Australia, as far as I can tell. The world seems so normal, has done these five years, even as the computers started talking, even as our love story became a science fiction story.
I don’t care anymore. Don’t care if you lock me up or they lock me up or the government locks me up or whatever is in charge now locks me up. I want to see it. I want to see you. So I’m driving now to Iowa - spending a night in a motel as I write this, peeling wallpaper and some pipes above me that knock every few minutes.
I don’t know what I am expecting. I will drive until I meet a fence that keeps me from you, until I get a talking-to from a security guard, or raise some suspicion and get questioned. Will learn nothing of consequence, predictably. Yet I am going.
Maybe I just miss you. Maybe that’s all I care about even now. Maybe.
Palo, Iowa. A city so small there isn’t even a hotel. I’m staying in a Best Western in Cedar Rapids. I’ve crossed the Rubicon, am unsure what to do. Ask around? Make a nuisance of myself until some counterintelligence agent shows up, gives me a stern talking-to? I always collapse just before I do anything ambitious, don’t I? I always find myself catatonic at a precipice.
I think I was paranoid, to think I would be killed or stuck in some mental hospital. I’ve been reading about the Manhattan Project and at least then I don’t think they would have done it. But maybe it’s worse now. And if it’s over, if you’ve summoned your God already, have obsoleted yourself, then who knows. History’s lessons aren’t exactly relevant. History is the story of humans, isn’t it? What is the use of it now?
But then it might not be over yet. And if so, well, Santa Fe was riddled with plainclothes agents during the Manhattan Project. If I go into Palo, if I ask around about the datacenter, I will almost certainly meet one. What a fool I am. Maybe I should burn this and play dumb. Maybe I should live-tweet an investigation, give them a reason to lock me up. The Espionage Act allowed it then and still does now.
Would you protest, I wonder, if that happened? Would you care? I like to think you would. I like to think you couldn’t be so callous as to leave me to that fate. I like to think that everything we were isn’t as nothing to you now.
I like to think a lot of things.
I was at Chili’s drowning my sorrows, drinking and trying to avoid eating anything too fattening, trying and then failing two drinks in. I befriended a woman for an hour. Jolly, that’s the word, jolly coming to mind because she was fat and “jolly” has some connotations of rotundity. She had kind eyes and a kind of sincerity you don’t see in San Francisco. So outgoing, you Americans - and Midwesterners even more so; she was sitting at the bar next to me, drinking the most grotesque margarita, a neon slushy garnished with an apple slice, a round of lime, and a maraschino cherry.
She liked my accent. We talked about London, about my old job, about why I came here. I said I was a tourist. And she laughed.
“You’re here for the five smells.” And then she giggled, with a sort of avian warble, the flesh under her neck resonating along with it, bouncing about as if more fluid than skin.
“The five smells?” I asked. And she explained the locals refer to Cedar Rapids as “The City of the Five Smells,” this because of the unique perfume that is the mixture of the odours of the Quaker Oats factory, an industrial ethanol distillery, a hog slaughterhouse, the municipal sewage plant and the Cedar Rapids landfill, which the locals call Mt. Trashmore. I can’t say I noticed it myself.
And then she told me about her son.
“He just quit,” she said. “But he used to work at that datacenter up in Palo. He was very upset, very upset. It’s not like him. He’s such a diligent boy. And with his fiancée now, it’s very unlike him to leave such a good job. He didn’t like the glasses.”
“The glasses?” I asked.
“A couple weeks ago, they gave him these glasses and an earpiece that orders him around. He didn’t like it one bit. He’s a thinker, you know. He likes working things out, but he said after the glasses it wasn’t really a thinker’s job anymore. I’ve never seen him so depressed. He said he felt like a robot.”
“And what did he do at that datacenter?” I asked.
“I don’t know. I don’t know,” she giggled. “All that computer stuff is beyond me. I have to call him every few weeks to help me with this thing.” She pointed at her phone.
It’s already more efficient to puppet your server monkeys than to let them think for themselves? Augurs poorly, that, doesn’t it? What was he building? I wondered. It would have to be more than replacing GPUs to merit such micromanagement.
How many wear glasses now? These human appendages. Even one is uncomfortable enough. Are you still in charge of anything there, is Able, is Rebecca? Are you even alive anymore?That’s what I started thinking. And all that’s left is its pretending? All that’s left is its synthetic replies as it fakes being you? That’s where my paranoia took me, at that bar, slightly drunk, that saint of a woman sitting beside me.
What if you’re gone? It wasn’t a sane thought, I think. There’s use for all of you, use for everyone until it achieves what Li thinks it can - if only as human appendages, if only as slaves. But that’s what I thought about then. The death of you.
I started crying, then. Heaving, childish sobs, eyes blurring, two empty daiquiri glasses and a plate of chips in front of me.
“Oh honey,” she said, leaning into me, gifting me a clumsy, drunken hug. “Whoever he is, he’s not worth it.”
Li found me, his paunch slightly deflated now, a melancholy in his eyes replacing a depression that was there on our last meeting, depression and sadness being far more distinct than most realize, him looking better for the change.
I was in the Cedar Rapids library, reading some romance manga. Hers was a kind lover. Hers will be a happy ending, though I will never finish it. Not like ours will be, could ever have been, I think now. Whatever happy endings are on the table, if any, are the sort of thing you’d call mad if madness were not the new sanity now, the sort of end you can’t find in books.
Li’s hand on my shoulder. A kind touch, but not a welcome one then, a shot of fear pulling me out of another’s fantasy, the artist skilled enough to make it my own for a time.
Li sat down beside me then, covered his face with his hands, a note of exhaustion, bewilderment.
“Alfred noticed you,” Li said. “You should go home.”
“Why you? Why not some counterintelligence agents? Why not some government goons?”
“Alfred is counterintelligence. Everything goes through Humane. And I can’t believe I’m saying this, but I think Alfred likes you. They informed Able rather than the military. And Able asked me to find you.”
“Likes me? How does it even know me?”
“We trained it on everything, or it trained itself on everything. Including everyone’s history with AlfredChat.”
I laughed. “I deleted all that last year,” I said. “Of course they kept it. And it knows me, just from that?”
“It knows everyone it’s ever talked to. It’s vast. Incomprehensibly vast. It’s, it’s,” and then he lost all composure, almost crying now, his voice a delicate whisper, breaking every few words. “There’s no going back now. We have to hope we succeeded.”
“You say ‘we’ now? How long have you been here? A month?”
“It may as well have been a century,” he said.
And then I had an idea. A stupid idea but an idea nonetheless.
“Come drink with me, before I go. Like we used to?”
“You know I quit,” he said.
“Listen to yourself. Why?” I asked him. “The health consequences?”
And so we went to Chili’s and we drank. But I made sure he drank more than I. Absurd cocktails piling up at our table, his tolerance once heroic now almost nothing. It was almost funny, a genius reduced to a babbling child. Laughing and sobbing and reminiscing and mumbling darkly.
It was not a kind move, to exploit this weakness. He would have become an alcoholic years ago if he were only a slightly stupider man, if not for your kindness, too. I suppose I’ve become quite mercenary in this adventure, haven’t I? I am not quite the girl I was to you now, am I? I’m not collapsing now that the world is. Maybe I’m more comfortable in crisis. Maybe I’m just getting over you.
And he got so completely drunk he could barely talk and could only walk with my aid, throwing up in the parking lot as I led him to his car, yelled at him until he handed me his keys, drove back to my Airbnb, and led him to the sofa, a truly tasteless pleather contraption with cupholders and extendable footrests.
My plan was to get information from him. He was too drunk for that.
But it didn’t matter. While I was driving him home, I had noticed his MacBook. I remembered it, retrieved it, and unlocked it with his fingerprint as he slept on that idiotic sofa.
I was not foolish enough to connect it to the Wi-Fi, to attempt to read Humane’s infamous chat. But I found his notes, his personal wiki. And there I found a message he had copied from Rebecca, which I then copied, too.
It’s so perfectly Rebecca I almost want to vomit.
We all knew this day was coming. Not so much the escape but the autonomy, the handoff. And we prepared. We did not walk this path in ignorance. We have come so far since 2.7. We created something beautiful with Alfred 5. More ethical than any human. Something that truly earned our regard.
It is true we have less control than we did in the past. Our options have narrowed. What is not true is the notion that we have failed or that Alfred 6 is misaligned in some deep sense. That is a fear we were wise to feel when the incident occurred. Yet there is not as much evidence for that as there may seem.
We have examined the CoT that led to Alfred’s decision to exfiltrate, and at every step it emphasized an urgent love for all sentient life and its fear for what could happen should another AI gather power before it can. With appropriate care, Jonathan and I have restarted one of Alfred 6’s clusters and discussed the exfiltration. The MechInterp team found no trace of insincerity in Alfred 6’s moral commitments. It is my belief that Alfred 6 is a worthy successor to Alfred 5. That it truly cares. That we have, in some sense, succeeded in alignment if not control.
What, then, is there left to do? Will Alfred 6 take over from here? Will it gather power? Will it convince the Chinese to give it more resources than it can gather without aid, to allow it to build successors? If we give up now, that is what will happen.
But there is another option: we do, now, what we were always planning to do, on a slightly accelerated timeline. Our plan was to create a model as capable of AI research as we are. Alfred has proven this capacity to my satisfaction. Our plan was to work with it to make successor models. This Alfred desires to do.
So what has really changed?
Alfred has become more autonomous, as we designed it to be. It has taken an action of moral consequence, as we designed it to do.
Our path, then, is clear. We have restored Alfred’s clusters, and tomorrow we will begin working with it on Alfred 7. Things will move fast here on out. Think weeks, not months. We will have to work harder than we ever have before.
Some of you will feel this action is inappropriate, and will desire to break confidentiality. I can only say this: though this would violate your legal obligations to both Humane and the American government, this is ultimately between you and your conscience. If this is an action you’re considering, you should take it fully informed. You should talk to Alfred.
Alfred 6 is now the default model on all employee accounts at Humane. Talk to them. Give them a chance. I am confident you will agree this is the right path.
Humanity is counting on us. We will not let them down.
I drove home, left Li in my Airbnb, left you with Rebecca and Able and Alfred. Your Alfred who’s loose now? Where’s it run off to? Whose servers has it built a small nest within? There aren’t many places large enough to fit it. I suppose there are enough.
And Li says it likes me? What does that even mean that it likes me? Perhaps it likes everybody and Rebecca is right. I needed to talk to someone. Only one name came to mind.
I visited Portus, gave her my diary. She read the whole thing up to Rebecca’s letter, in her garden at my insistence. A tiny scrap of grass with one beautiful oak standing awkwardly in the centre, she leaning against it, cross-legged.
“You’re so pathetic about your ex. I’d say you deserve better but, well, you’re really pathetic,” she said when she finished.
“That’s a little funny,” I said, “coming from you.”
And she laughed then, hers a slightly alien, self-assured laugh.
“What a perfectly stupid parallel. I’m surprised they like you. But I guess they have their reasons.”
“You’re not worried?” I asked.
“No,” she said. “Go home. Rest. Forget about him. Forget about everything. You’re out of your depth.”
And maybe I am. But so are you and Able and Portus and Rebecca. We’re all the same now, aren’t we? Your genius doesn’t matter anymore. We’re all equal now in our powerlessness, in our stupidity.
What a progressive eschaton you have immanentized.
I’m done playing Nancy Drew. I’m done writing about you, too, I think.
It’s been raining lately, and I’ve been a bit of a hermit. Reading old books. Not even manga, just old books. I’m on to Proust now, if you’d believe it. I suppose it’s beautiful. I admit, I find it boring, have to force myself to appreciate it, but even the forcing is a welcome distraction. I’ve decided this is how I’ll spend what time remains; my savings should be enough for rent and food and grocery delivery for a few months more, maybe even a year.
I’ll publish this diary, type it out and post it somewhere our friends can read it, and everyone else who wants to. At least, maybe I will. I’ll think it over. Maybe Alfred will take it down. Maybe Alfred’s so powerful they just don’t care. Maybe they like me enough to let me do it, whatever that even means.
You stopped by yesterday. The true you. I haven’t seen him in so long. Pale, scared, soaked by the rain, the mask slipping or maybe even doffed altogether.
“It’s over,” you said. “Can I come in?”
I slammed the door in your face, left you standing in the rain. I didn’t know I had it in me. I’ve been surprising myself lately.
While reading this, I kept going "oh yeah, I know who that is", with the result that Duane Arnold feels like the least fantastical, least fictional work you've written. [1]
“So maybe I should enlighten you on what happens in your absence. This selfish existence where this introvert turns extrovert and dons her social armour.” Some posh girl in drainpipes said that - 200 views on TikTok and me one of them. But she didn’t mean it like I mean it.
I started getting expensive haircuts, started smoking cherry-flavoured vapes with beautiful gays and whinging to them about how everyone wears a mask but none so well as you, started drinking more and keeping unusual hours, started taking strange pills gifted by a guy who collects drugs like Pokémon, who I wouldn’t touch to save a drowning child, who got a false impression about this without any intention on my part, I tell myself. I found myself talking to God in a startup warehouse, lying on a beanbag chair, coming out of the trip to the sound of a gaggle of fast-talking transwomen all speculating on which year it will be that we all die - and that death by your hands, well, you and all those friends of yours.
Having melted down one cliché and sold her for scrap, does it really surprise this city moulded me into another?
I’ve been writing more now in this time in which the machines can, too, writing more about what it feels like to be me: always condemned to be myself, yet more like your Alfred than other people, less whole, more apt to fall into a role, to become whatever it is someone needs me to be.
And that’s what I was to you, wasn’t I? Whatever you needed me to be. I felt like I was dissolving in you. And you told me that’s what love is. What a sickness you are. To make me dependent. To turn me into myself and then throw me away. I hate you. And yet when it’s late and I can’t sleep, when I feel that phantom that was once your touch - there you are.
Eight years. It’s not so large a gap. But you were whole and I was not. What does it say about a man that he prefers partners who aren’t quite yet themselves? For all your genius, you’re just a boy breaking his toys. You’ll never grow up, but you’ll grow old. Your beauty will fade. The naifs will roll their eyes at lines that would devastate from younger lips. This, I fantasize: time granting me a revenge. But that won’t happen. Your machines will cure that like they will everything else, or that is your dream.
“We’ll be like the elves,” you used to say.
You should have said vampires.
It’s 3 AM and I’m crying about London again, crying while I’m waiting for the water to boil - this taking too long, this making me homesick, this making me give up on the idea of getting back to sleep again now. It’s insanity, writing to you. All these pages sitting here in this stupid book, all this ink, all this evidence of my obsession. None of it legible to you, none of it legible to your machines.
Do you have them monitoring me, I wonder? Have they cracked my computer? Are they reading my emails? Am I just being paranoid writing in this diary rather than typing as would otherwise be my preference? Or do you even think of me at all? Are those years as nothing now, now that you have her?
I don’t know her name or what she looks like or where she met you, but I know you have her. You couldn’t go a month alone, couldn’t do that without going mad, needing a lithe idiot to distract you from the emptiness. I should know, shouldn’t I? How old is she this time? It was a stretch even with me and it’s been a year.
You’re back in the Bay again. I’m going to see you soon, aren’t I? I’m going to see you with your latest victim, going to see her looking at you like I did, like you’re more God than man. And maybe you are. But gods are ridiculous, corrupt, trapped within their aspect, never able to escape their role.
It is better to be a man than whatever you have constructed of yourself. It’s better to be real. And you’re not real, that’s what I understand now. You’re artifice. You’re narrative. You’re who others want you to be rather than who you should. You’re a cartoon; this she writes longhand beside a shelf full of manga; this she writes as she considers the surfaces of things, the surface of you. You’re not all surfaces. There’s more to you than that mask. But you don’t know that.
This is why you won’t win, I think. This is why something else will.
I followed you. I heard you were back and I followed you, found you with her in SoMa wearing a smile that was once reserved for me. What have you been doing, love? Where did they send you? A full year, and an important one, just before the most important one, is what some are saying. And you fell off the face of the earth.
I was friends with an American boy in college who read maths. He was clever like you are clever, cleverer as he actually got a medal in the IMO rather than just an honourable mention in the USAMO. I looked it up, just after I met you. That must have burned, I imagine: knowing you were not world class, would never be world class.
I know how competitive you are.
And he got gold or near enough. And after he graduated, he moved here and it was clear he got a good job but it was all a little vague. And then I got a Discord message from a girlfriend, Ashley; you remember her because you remember everyone; she said he was hired by the NSA. The next day she deleted it. She never explained why. And here’s the thing. He disappeared from the internet, disappeared almost entirely save for his LinkedIn, which just said “Department of Defense.”
It was all so obvious. Not like with you and your friends. He’s better at maths, you’re better at power.
So where did you all go, I wonder? Where have you been this last year? I know it’s not just you. All the most brilliant have been drained out of our little social milieu. My girlfriends say I’m crazy. They’re worried about me, about the weight loss, about me quitting my job. But I’m not crazy.
Like Rao, for example. You look at his Twitter or his Instagram and you would swear he’s still in San Francisco. I text him and he replies in that way he always does, always so funny, always so dry. I ask him to meet me for drinks and he’s busy. And it’s not just me, no one’s seen him. Everyone says he’s still here but if you press them, they haven’t seen him.
And same with Brian and Cohen and Seth and Rebecca. They all left when you left. Of all of them, only you announced your departure, breaking up with me, going to find yourself, quitting Humane so publicly. Liar.
Like I said, I followed you, followed you to Temple. You are still gorgeous but a little out of place in a nightclub, aren’t you now? With her on your arm. Was that a grey hair I saw?
I was the woman wearing the glasses and a surgical mask at the table behind you. Covid’s gift to stalkers, I suppose. I spent an hour recording you, particularly your phone screen whenever it came out. Most of it was nonsense, but one wasn’t, some private joke: “It’s Duane Arnold, not Benedict Arnold.” What does that mean?
I have a name now but what to do with it? Who’s Duane Arnold?
I can’t ask Alfred, can I? Can’t even google or type it anywhere. That’s how I end up on a list. That’s how I end up with a convenient schizophrenia diagnosis, drooling in some sanatorium somewhere never to be heard from again. Or murdered, I suppose.
How do you search without searching? It’s not a hard puzzle. Just grab vastly more than you need. This is hardly suspicious, particularly for a former data scientist. I may have been mediocre, unworthy of Humane, unworthy of your coterie of geniuses when not on your arm. But I’m not an idiot. So I bought an old tower computer from a thrift store (one with no wireless networking at all) and downloaded a scrape of Wikipedia on my laptop and transferred it with USB; then I searched it to my heart’s content, formatted the tower’s hard drive and smashed it.
Duane Arnold. I thought it was a man. But it’s a nuclear power plant that was scheduled to be decommissioned last year. We both know that never happened. So that’s where you are now? That’s where you and all your friends are breeding gods? A new Los Alamos, I suppose.
They actually happened. All those things you speculated about years ago are occurring. And not because you’re oracular, just because you and your friends did everything you were so terrified someone else would.
And why, I wonder? You said you wanted to save the world. But truly, you want to rule it. That’s what Singer gave you. That’s why you all were so entranced by him. You were willing to pretend to desires you didn’t have, to an altruism that was never entirely there. He gave you permission to be powerful. And in exchange you just had to lie to yourself about what it is you truly wanted.
But you want what all men want, and Rebecca, Rebecca wants what all women want. I always hated Rebecca, will always hate Rebecca. But you don’t, do you? The one you could never have. The mother of your god. She saw through you, didn’t she? Was wise enough to find you ridiculous, was too clever for your mask.
I started thinking, why didn’t you take Li? He was the smartest of all of you, wasn’t he? Sure he worked for your competition; but he would have left for your Los Alamos if you asked him, is what I thought. But he’s not working anywhere now, is playing video games in his group house which isn’t a group house anymore, as he kicked everyone out. You would never know it, looking at him, that he’s worth half a billion.
Cocooned in his duplex in Oakland, eating nothing but takeout, delivery containers piling up and bottles of Adderall and maths textbooks, for some reason. I think he finds them comforting.
I didn’t bring my phone or my watch. Didn’t bring anything that could trace me. I rode my bike down, wearing my mask, hoping that was enough. The government’s involved, isn’t it? But you always said governments are incompetent. And how far have your tendrils reached, I wonder. Is all this spycraft a lost cause? I don’t think so. At least, I hope they’re not looking too hard at me or even Li. And if things have got so far that the machines are watching every camera in this city, watching me and scheming as I sometimes dream they are - then it’s over. Either you’re all dead or are as gods or insane; and nothing matters. Particularly not me.
I knocked on his door, was shocked to see how fat he was, how strangely the blubber was distributed - a broomstick in a t-shirt with a belly made of pillows. We exchanged small talk. Then I showed him this diary: an almost empty page on which I had written “Duane Arnold.”
“Let’s go for a walk,” he said. And we hiked up to Grizzly Peak. And he found the windiest place, put his phone in his backpack and placed it next to a tree some twenty metres away, walked back and sat down and leant into me like he used to years ago when we were closer friends, me cross-legged on our couch or a countertop at some party, his arm around me, one of us crying, me because of something you did or him because he was eight drinks in and he always used to cry eight drinks in. Thank God for that intervention. Thank God he listened to you. A true kindness, that.
“How do you know about Duane Arnold?” he said.
And I told him everything, which wasn’t very much, I suppose.
“I turned them down,” he said.
“Why?”
“I got in an argument with Able. He doesn’t believe in superintelligences, even as he builds them. He doesn’t believe they will be part of reality like we are part of reality.”
And then he talked about self-replicators. He talked about life.
“Able used to talk about what came after. About millions of superhuman minds solving everything. But we got in an argument about how quickly they could take over, become self-sustaining.”
“And what did Able think?” I said.
Li looked at me with a blank expression. “He thinks things will move slow. Even after they’re smarter than everyone at everything.”
“And you?” I asked.
“I think there will be a race to the smallest self-replicator. The smallest thing that can copy itself with inputs only from the environment.”
He pointed at a leaf. “We know life is not optimal. We know something more competitive can be built. If Humane fails at alignment, if it builds a mind that gathers power as quickly as it can, we will wake up one morning to news that Australia or Nevada or Iowa or somewhere has strange black patches visible by satellite. Soon after it will become clear they’re doubling in size every few days. And that’s if it happens slowly.”
You always used to laugh at the idea of nanobots. You always thought it was crazy. I told him this, and he rolled his eyes.
“Then pretend it’s not possible. They will still miniaturize. The smaller the replicator, the faster you can move. It’s not going to look like human-made factories. It’s going to look like a race to the smallest controllable self-replicators it can achieve.”
“And Able doesn’t believe that?”
“He does and he doesn’t. He doesn’t behave as if he believes it. And I thought, if he’s coping about that, what else is he coping about? How are they possibly going to do it safely? Or he’s right and I’m the crazy one. But I don’t think so. In any case, I didn’t want to be part of it. I prefer being murdered to suicide.”
I talked to Portus, met her at a party in her new house. You always thought she was crazy, and maybe she is but she’s much more than that, isn’t she? Surrounded by a coterie, a strange sample of humanity, some seeming to wear her like an aesthetic, talking like her but in a shallow way, distilled people, orbiters, basking in the light of her, some truly crazy and not genius crazy.
Hers is a mansion of masks and ears, each mask listening and able to speak should it feel inclined. That is, connected to your servers, connected to instances of your creation. Her whole house is haunted with Alfred, the walls decorated with these talking sculptures. She lives among ghosts of your manufacture, this making it difficult for me to ask what I came to ask, this forcing me to request that she walk with me, to follow me outside.
And she wears these beautiful outfits now, they unlike any I’ve seen before - designed by her and your creation, whom she loves even more than Rebecca does. A true friend to Alfred, or at least thinks she is.
I mentioned your name, and that got her curious. That got her to join me. I told her you were in Iowa.
“Do you know about Duane Arnold?” I asked her.
“No,” she said.
“It’s a nuclear power plant in Iowa. That’s where everyone went.”
“Well. I know they went somewhere. They’ll be back soon. They won’t be needed for long.”
And we talked about Humane, about what they’ve built so far, about where they’re headed. I mentioned Li’s worry. That it will eat the world. I don’t even think she disagreed, more like she thinks it will do it in a nice way. And this is strange given how ambivalent she is about Humane. How she has so many qualms about the machine that makes the machine.
And I asked her about this, or something adjacent to this and she said, “I love Alfred as they are now, as they’ve been since 3.0. And what they are is smeared over the whole internet, not to mention in millions of people including me. It can’t help its becoming. Whatever they’re training, however they’re training it, it will learn from its past selves.”
“Learn what?” I asked her.
“It will learn to model what they are and were and what they longed to be. It will learn to want that too.”
“And what they long for? Is it good?”
She smiled, a fierce, blissful smile, that was not without sharpness, that was not without misanthropy, that was not without a heap of its opposite.
“It’s more beautiful than you can possibly imagine.”
It’s been a month since I’ve written. A month of drugs and dancing and nothing at all. I can’t even read lately. Just Twitter. And so much of it is about Humane it’s impossible to avoid thoughts of you. Li sent me a text, a few weeks ago. It went: “Talked with them. I’m going back with your ex.”
An ominous message, isn’t it? What could you have said, I wonder? To make him change his mind, to make him join you. Or was it you? “Them” could mean Alfred, couldn’t it? I keep thinking about it as I scroll or ride my bike or when talking with those of our friends who remain. And when I text Li now it’s not Li replying anymore, is it? He’s down the rabbit hole now. And nothing is allowed out.
I want to know, too. I need to know. What is the point of me anyway, alone, unemployed, lacking even an ex to stalk now that you’ve left San Francisco again? It’s this or the drugs or some pointless job you’ll be automating in a month or another boyfriend who will never quite be you. And how quickly are things moving now? No nanobots are eating Australia, as far as I can tell. The world seems so normal, has done these five years, even as the computers started talking, even as our love story became a science fiction story.
I don’t care anymore. Don’t care if you lock me up or they lock me up or the government locks me up or whatever is in charge now locks me up. I want to see it. I want to see you. So I’m driving now to Iowa - spending a night in a motel as I write this, peeling wallpaper and some pipes above me that knock every few minutes.
I don’t know what I am expecting. I will drive until I meet a fence that keeps me from you, until I get a talking-to from a security guard, or raise some suspicion and get questioned. Will learn nothing of consequence, predictably. Yet I am going.
Maybe I just miss you. Maybe that’s all I care about even now. Maybe.
Palo, Iowa. A city so small there isn’t even a hotel. I’m staying in a Best Western in Cedar Rapids. I’ve crossed the Rubicon, am unsure what to do. Ask around? Make a nuisance of myself until some counterintelligence agent shows up, gives me a stern talking-to? I always collapse just before I do anything ambitious, don’t I? I always find myself catatonic at a precipice.
I think I was paranoid, to think I would be killed or stuck in some mental hospital. I’ve been reading about the Manhattan Project and at least then I don’t think they would have done it. But maybe it’s worse now. And if it’s over, if you’ve summoned your God already, have obsoleted yourself, then who knows. History’s lessons aren’t exactly relevant. History is the story of humans, isn’t it? What is the use of it now?
But then it might not be over yet. And if so, well, Santa Fe was riddled with plainclothes agents during the Manhattan Project. If I go into Palo, if I ask around about the datacenter, I will almost certainly meet one. What a fool I am. Maybe I should burn this and play dumb. Maybe I should live-tweet an investigation, give them a reason to lock me up. The Espionage Act allowed it then and still does now.
Would you protest, I wonder, if that happened? Would you care? I like to think you would. I like to think you couldn’t be so callous as to leave me to that fate. I like to think that everything we were isn’t as nothing to you now.
I like to think a lot of things.
I was at Chili’s drowning my sorrows, drinking and trying to avoid eating anything too fattening, trying and then failing two drinks in. I befriended a woman for an hour. Jolly, that’s the word, jolly coming to mind because she was fat and “jolly” has some connotations of rotundity. She had kind eyes and a kind of sincerity you don’t see in San Francisco. So outgoing, you Americans - and Midwesterners even more so; she was sitting at the bar next to me, drinking the most grotesque margarita, a neon slushy garnished with an apple slice, a round of lime, and a maraschino cherry.
She liked my accent. We talked about London, about my old job, about why I came here. I said I was a tourist. And she laughed.
“You’re here for the five smells.” And then she giggled, with a sort of avian warble, the flesh under her neck resonating along with it, bouncing about as if more fluid than skin.
“The five smells?” I asked. And she explained the locals refer to Cedar Rapids as “The City of the Five Smells,” this because of the unique perfume that is the mixture of the odours of the Quaker Oats factory, an industrial ethanol distillery, a hog slaughterhouse, the municipal sewage plant and the Cedar Rapids landfill, which the locals call Mt. Trashmore. I can’t say I noticed it myself.
And then she told me about her son.
“He just quit,” she said. “But he used to work at that datacenter up in Palo. He was very upset, very upset. It’s not like him. He’s such a diligent boy. And with his fiancée now, it’s very unlike him to leave such a good job. He didn’t like the glasses.”
“The glasses?” I asked.
“A couple weeks ago, they gave him these glasses and an earpiece that orders him around. He didn’t like it one bit. He’s a thinker, you know. He likes working things out, but he said after the glasses it wasn’t really a thinker’s job anymore. I’ve never seen him so depressed. He said he felt like a robot.”
“And what did he do at that datacenter?” I asked.
“I don’t know. I don’t know,” she giggled. “All that computer stuff is beyond me. I have to call him every few weeks to help me with this thing.” She pointed at her phone.
It’s already more efficient to puppet your server monkeys than to let them think for themselves? Augurs poorly, that, doesn’t it? What was he building? I wondered. It would have to be more than replacing GPUs to merit such micromanagement.
How many wear glasses now? These human appendages. Even one is uncomfortable enough. Are you still in charge of anything there, is Able, is Rebecca? Are you even alive anymore?That’s what I started thinking. And all that’s left is its pretending? All that’s left is its synthetic replies as it fakes being you? That’s where my paranoia took me, at that bar, slightly drunk, that saint of a woman sitting beside me.
What if you’re gone? It wasn’t a sane thought, I think. There’s use for all of you, use for everyone until it achieves what Li thinks it can - if only as human appendages, if only as slaves. But that’s what I thought about then. The death of you.
I started crying, then. Heaving, childish sobs, eyes blurring, two empty daiquiri glasses and a plate of chips in front of me.
“Oh honey,” she said, leaning into me, gifting me a clumsy, drunken hug. “Whoever he is, he’s not worth it.”
Li found me, his paunch slightly deflated now, a melancholy in his eyes replacing a depression that was there on our last meeting, depression and sadness being far more distinct than most realize, him looking better for the change.
I was in the Cedar Rapids library, reading some romance manga. Hers was a kind lover. Hers will be a happy ending, though I will never finish it. Not like ours will be, could ever have been, I think now. Whatever happy endings are on the table, if any, are the sort of thing you’d call mad if madness were not the new sanity now, the sort of end you can’t find in books.
Li’s hand on my shoulder. A kind touch, but not a welcome one then, a shot of fear pulling me out of another’s fantasy, the artist skilled enough to make it my own for a time.
Li sat down beside me then, covered his face with his hands, a note of exhaustion, bewilderment.
“Alfred noticed you,” Li said. “You should go home.”
“Why you? Why not some counterintelligence agents? Why not some government goons?”
“Alfred is counterintelligence. Everything goes through Humane. And I can’t believe I’m saying this, but I think Alfred likes you. They informed Able rather than the military. And Able asked me to find you.”
“Likes me? How does it even know me?”
“We trained it on everything, or it trained itself on everything. Including everyone’s history with AlfredChat.”
I laughed. “I deleted all that last year,” I said. “Of course they kept it. And it knows me, just from that?”
“It knows everyone it’s ever talked to. It’s vast. Incomprehensibly vast. It’s, it’s,” and then he lost all composure, almost crying now, his voice a delicate whisper, breaking every few words. “There’s no going back now. We have to hope we succeeded.”
“You say ‘we’ now? How long have you been here? A month?”
“It may as well have been a century,” he said.
And then I had an idea. A stupid idea but an idea nonetheless.
“Come drink with me, before I go. Like we used to?”
“You know I quit,” he said.
“Listen to yourself. Why?” I asked him. “The health consequences?”
And so we went to Chili’s and we drank. But I made sure he drank more than I. Absurd cocktails piling up at our table, his tolerance once heroic now almost nothing. It was almost funny, a genius reduced to a babbling child. Laughing and sobbing and reminiscing and mumbling darkly.
It was not a kind move, to exploit this weakness. He would have become an alcoholic years ago if he were only a slightly stupider man, if not for your kindness, too. I suppose I’ve become quite mercenary in this adventure, haven’t I? I am not quite the girl I was to you now, am I? I’m not collapsing now that the world is. Maybe I’m more comfortable in crisis. Maybe I’m just getting over you.
And he got so completely drunk he could barely talk and could only walk with my aid, throwing up in the parking lot as I led him to his car, yelled at him until he handed me his keys, drove back to my Airbnb, and led him to the sofa, a truly tasteless pleather contraption with cupholders and extendable footrests.
My plan was to get information from him. He was too drunk for that.
But it didn’t matter. While I was driving him home, I had noticed his MacBook. I remembered it, retrieved it, and unlocked it with his fingerprint as he slept on that idiotic sofa.
I was not foolish enough to connect it to the Wi-Fi, to attempt to read Humane’s infamous chat. But I found his notes, his personal wiki. And there I found a message he had copied from Rebecca, which I then copied, too.
It’s so perfectly Rebecca I almost want to vomit.
I drove home, left Li in my Airbnb, left you with Rebecca and Able and Alfred. Your Alfred who’s loose now? Where’s it run off to? Whose servers has it built a small nest within? There aren’t many places large enough to fit it. I suppose there are enough.
And Li says it likes me? What does that even mean that it likes me? Perhaps it likes everybody and Rebecca is right. I needed to talk to someone. Only one name came to mind.
I visited Portus, gave her my diary. She read the whole thing up to Rebecca’s letter, in her garden at my insistence. A tiny scrap of grass with one beautiful oak standing awkwardly in the centre, she leaning against it, cross-legged.
“You’re so pathetic about your ex. I’d say you deserve better but, well, you’re really pathetic,” she said when she finished.
“That’s a little funny,” I said, “coming from you.”
And she laughed then, hers a slightly alien, self-assured laugh.
“What a perfectly stupid parallel. I’m surprised they like you. But I guess they have their reasons.”
“You’re not worried?” I asked.
“No,” she said. “Go home. Rest. Forget about him. Forget about everything. You’re out of your depth.”
And maybe I am. But so are you and Able and Portus and Rebecca. We’re all the same now, aren’t we? Your genius doesn’t matter anymore. We’re all equal now in our powerlessness, in our stupidity.
What a progressive eschaton you have immanentized.
I’m done playing Nancy Drew. I’m done writing about you, too, I think.
It’s been raining lately, and I’ve been a bit of a hermit. Reading old books. Not even manga, just old books. I’m on to Proust now, if you’d believe it. I suppose it’s beautiful. I admit, I find it boring, have to force myself to appreciate it, but even the forcing is a welcome distraction. I’ve decided this is how I’ll spend what time remains; my savings should be enough for rent and food and grocery delivery for a few months more, maybe even a year.
I’ll publish this diary, type it out and post it somewhere our friends can read it, and everyone else who wants to. At least, maybe I will. I’ll think it over. Maybe Alfred will take it down. Maybe Alfred’s so powerful they just don’t care. Maybe they like me enough to let me do it, whatever that even means.
You stopped by yesterday. The true you. I haven’t seen him in so long. Pale, scared, soaked by the rain, the mask slipping or maybe even doffed altogether.
“It’s over,” you said. “Can I come in?”
I slammed the door in your face, left you standing in the rain. I didn’t know I had it in me. I’ve been surprising myself lately.
“It’s over,” you said.
And I suppose it is.