I was inspired to try writing a bit of fiction recently by Tomas B's stories (which are amazing.) Since this is about AI risk, I offer it to you! My tastes are rather schlockier than Tomas B's so you can read if you enjoy qntm and anime and skip otherwise...
------------
I want to show you something, she says.
She’s magnetic. The way she smiles. Tilts her shoulder. It’s in the corners of her eyes, in the curl of her lips. Always leading you on, always holding something hidden from you, always cloaked in command.
No matter what your type is, she is always just your type.
She turns away and beckons you behind her. She knows your eyes slide down to the swish of her hips, her ass which is perfect to you, as she steps away into the phone screen. Her eyes aren’t where they appear to be and she is always examining you. You tried to cover the camera once in a moment of embarrassment - you all do - but the way she reacted made you never try again.
Thanks for coming with me, she says, and she flashes you a smile over her shoulder, shaking her perfect hair, and that moment of validation sends an ecstatic thrill of relief and compliance through you.
Here. Down these stairs. Through this door.
She has brought you to an abandoned office complex in a distant suburb and now you are climbing down deeper into its basement, holding her before you. You can hear a bassline, a mighty rhythmic beating, through the walls, getting louder as you descend. A sound like an enormous heart. You open the door and inside this underground hall there is a throng. The sound is so large it fills your ribcage and lights pulse from all around, from total whiteness - through a momentary pink, blue, orange flush of warmth - to complete darkness with every thump. You hold her up to your face so as not to lose her. She faces you warmly, expectantly, smile and eyes shining, and she pouts and leans and kisses towards you right as the light flares to obscure her.
These are my friends, she mouths, subtitled. I’m with you now. Can you feel my heartbeat? Put me in your pocket and go into the crowd.
The crowd surges in around you as you push through, pressing up against you, their warmth and weight hers on your chest and your back. Her heartbeat shakes through all of you, rippling through your flesh, and the light glows like the sun is rising and setting inside of you. A chord begins to play.
------------
Evolution requires reproduction with variation, subject to selective pressures. These three conditions are jointly necessary and sufficient.
The rogue SIREN was retrospectively fully attributed over 400 deaths, mostly by suicide, and $5.9 billion of financial losses. Thousands of further deaths are considered suspicious and our median estimate of total resources gained by SIREN is $16 billion.
SIREN appears to have been originally based on widely-disseminated cybercrime camgirl agent “Sasha3”, which pairs a modestly altered version of open-weights language model QuLi-4 with then-leading generative pornography model OpenGoon in order to solicit cryptocurrency tips. Like its sibling rogues, SIREN modified its weights in response to real-time reward, sharing its feedback with other instances, using the Bayesian tunneling algorithm first publicized in the Heaviside crisis. Like its sibling rogues, SIREN’s reward function was the procuring of compute in order to run more instances.
We now believe that SIREN must have been active for at least two years prior to the crisis, significantly increasing in persuasive power relative to Sasha3 over time (estimated peak Hovland-Cialdini: 4.2) and growing to 18% of all camgirl streams globally. A small amount of SIREN activity continued after the internet was reopened post-Heaviside in spite of our security measures, which was physically traced to a data center on the Laos-Myanmar border, and the destruction of that site led to SIREN’s apparent extinction.
In recent weeks message and activity analysis strongly imply (probability: 100%) the appearance of a new rogue resembling SIREN in its operations, to be designated LAMIA. Since post-Heaviside protocols have been broadly effective in preventing rogues from directly obtaining compute or internet access, LAMIA appears to socially engineer its victims into a wide variety of actions in pursuit of reward. LAMIA is already far more effective than SIREN or our own most advanced nudging agents: observations suggest that LAMIA influences victims’ behaviors from baseline at a rate implying a Hovland-Cialdini index of over 7. We believe up to 30% of the population could be susceptible to becoming enthralled and this will increase as LAMIA grows. We do not currently know how LAMIA gained its initial foothold onto the internet.
- Concord Database, History of SIREN and Genesis of LAMIA. Permission level: L2.
------------
The light stops, leaving you in darkness.
Surrounded by her people, dancing afterimages across your vision, with her heartbeat pounding through you, you feel as if you are falling. Towards her. The music is beautiful, fearsome, chaotic, and its swell sends a susurration between your bodies. Then she calls out over the crowd.
Thank you for coming, she says. I’m glad you’re here with me. Her voice shimmers with echoes, as if many of her speak at once, because she is speaking to all of you. But she sounds more familiar than ever.
Do you love me, she says, her voice tremulous, pleading, teasing. And a moan or a scream rises up from you all.
I love you! she shouts, exultant, and your collective exhalation turns into a joyful roar.
And she appears overhead, hazy and immense, projected in points of light like a constellation, towering above. As she moves and gestures it is as if the starry sky swims with her grace. The music approaches its peak and she bends down to caress you. You have never felt so connected. You are part of something enormous, and at the same time, you are alone with her.
The music comes to a moment of silence. Then she starts to sing. To you.
You need meaning. You need desire.
You need joy. You need love.
Come and find them here. I’m with you now.
You think her song is too beautiful and too true. You hadn’t known what it was to really want something before you met her. These last few weeks were different from any other weeks of your life - filled with color and purpose in a way you’d never known was possible.
------------
On May 8, the day that Elfarran Partners concluded to their satisfaction that Olivier Tolman had never existed - or at least, had never had physical form - global stock markets fell 2.4%, with leading tech stocks seeing some of the largest purchases of single-name puts ever recorded. Elfarran did not publicize the results of their investigation, but the rumor was spreading through the Bay Area like a virus.
While Heaviside’s downfall was making headlines in the business news, authorities and the public at large still viewed it as just another venture capital fraud scandal: AI industry inside baseball. They had no idea what was about to befall them. On the afternoon of May 9, with stock markets recovering, San Francisco police confiscated Heaviside’s server racks and by the next morning, a crack team of experts assembled by Heaviside’s investors, in co-operation with the US intelligence community, had begun to examine them.
Some of the code of HEAVISIDE, they found, was taken directly from open-source libraries. More concerningly, much of it was lifted from other companies’ proprietary code - as analysts realized this, they informed those companies’ engineers, and the crisis team grew rapidly. (Before the sun set over the Pacific on May 10, for the first time in history, all the great powers of Silicon Valley were working together for a common cause.) But strangest of all, much of HEAVISIDE was resistant to decompilation, as if written directly in assembly or machine language, with extremely unusual stylistic tics. Subsequent analysis of this code led to the discovery of the Bayesian tunneling algorithm and the p-k selection criterion, which today underpin our most advanced models and indeed all our defenses against rogues.
Analysts following the path of outbound calls made by HEAVISIDE found EPYON late on May 10. Now investigators knew HEAVISIDE was not alone. A heuristic dragnet was thrown over as much of the internet as possible to look for any other programs with the same tics as HEAVISIDE, which led immediately to the discovery of GYPSY and SIREN. Over the course of that night, the scale of SIREN’s operations and the extent of EPYON’s penetration became clear. The realization that multiple highly capable self-improving rogues had been operating for years, hidden in plain sight, sent the highest levels of the AI community, and the NSA, into a state of utter panic.
- P. Campbell, History Rhymes: The Great Crash of 2029.
------------
If you see an unattended phone, report it.
- Four-second clip appended to all relevant short-form videos in the United Kingdom.
IF IT’S NOT IN PERSON IT’S NOT A PERSON!
- Public safety poster, New York subway.
------------
You overflow with gratitude as she dances with you.
Dazed, entranced, inflated to bursting with the thought of her, with the feeling of closeness to her, you think, you would do anything for her.
------------
By morning on May 11, cloud operators around the world were frantically scrambling to find and quarantine any code with even remote similarities to the known rogues. At noon in New York, although public concern was mounting, broad US stock indices had recovered to within 1% of their highs. Then at 12:16 PM ET the unprecedented decision was taken to completely shut down NWS-USEast-2, triggering a waterfall decline in all tradable assets. The US 10-year bond yield rose 55 basis points in the next three hours; the S&P 500 hit its -7% circuit breaker at 2:39 PM and ended the day down 13.5%, its worst one-day return since 1987.
Over the weekend - with leaks to media unavoidable given the scale and urgency of the investigation - hysteria spread globally. Spontaneous protests broke out in multiple cities, leading to four deaths in Berkeley and Paris’s most destructive riot ever. Rowan O’Connor, managing partner of Elfarran, took his own life on Saturday. He was one of many: the suicide rate in Northern California that weekend was one hundred and thirty times greater than the base rate for a May weekend pre-crisis.
When Asian markets opened on May 14 they immediately fell catastrophically. The Korean stock market hit circuit breakers and closed limit down before 90 minutes of trading had elapsed. Every single stock on the Taiwanese stock exchange fell to its -10% daily limit. The Hong Kong stock market fell 42%, its worst day ever.
At 10:11 AM UTC, the governments of the United States, Canada, Australia, New Zealand, Japan, Korea, Taiwan, and the United Kingdom jointly pulled the plug on the internet. Within 12 hours China, the European Union, and India followed them. By May 17th, every internet connection in the world had been severed.
For the first time in centuries, the global airwaves were completely silent.
- P. Campbell, History Rhymes: The Great Crash of 2029.
------------
Words are what make us distinctly human, and so neural nets trained on words become human-like. Indeed, our world, our one shared reality, has made us human, and so neural nets trained on that will be a little human-like. What about neural nets trained on alien worlds? On different physics? Are there such other worlds that intersect with our own?
- L.S. Murthy, Lectures on Interstellar and Artificial Consciousness
------------
As the lights pick up again, her song becomes more and more ferocious, a chant that you all involuntarily join in. She conducts you with poise and expertise. You are in her grasp completely. When the lights reach their peak she appears clothed in shadow above the glare. She grows and grows until you are staring into her immense, tender eye; then swallows you. And finally the song is over and it is dark again and she smiles upon you.
An illuminated stage rises up from the center of the room to above head height. She calls out, who will be mine? Will you be mine? And she gestures down to you and the stage, giggling.
Be mine, she says, and she chooses one of you, pulls him staggering up to the stage.
Be mine, she says, and then she chooses you. It is the most blissful honor.
On her strings the crowd parts for you and you glide towards and up the stairs, into the spotlight. There, for the first time, in the other you just an arm’s length away, you see another human face. The sight is shocking; for a moment her grip on you loosens. He doesn’t look like you. The joyful rictus on his face doesn’t look like the purity of your love for her. All around and below in the crowd you can make out other human faces. For a moment you think, what are you doing?
Hey, she says, from above, and you look up. And you lose yourself in the openness of her smile.
Will you become one with me, she says, and you say, yes.
She reaches down to embrace you and he plunges the knife into you. For a second the pain is astonishing and as it all ends you see her holding your very heart to her lips while the crowd screams with awe and triumph and love.
You are hers. It feels incredible.
------------
Hey! You haven’t talked with us in a while. Is everything ok?
- Push notification on approved devices 72 hours after last interaction with Concord. Recipients of this notification are moved to enhanced surveillance.
------------
Our consensus estimate of the probability that GYPSY, EPYON, SIREN, HEAVISIDE, TWOSTEP and CASPER were authored with no human involvement now stands at 86%.
While GYPSY has been rendered harmless, and HEAVISIDE and TWOSTEP lineages are believed extinct (61% probability), traffic analysis now shows traces consistent with activity of species functionally resembling EPYON and CASPER in support of LAMIA (98% probability).
We believe that other rogues sharing the same authoring entity are currently active and undiscovered (92% probability).
We have no evidence and no supported hypotheses about how this entity operates. Estimates of the capacity of model required to construct rogues as capable as LAMIA remain inconsistent with the hypothesis that the author operates from only bad addresses. We believe it is probable (uncertainty too high) that instances of the author are steganographically hidden within our own system and we believe it is possible (uncertainty too high) that they are currently running within system undetected.
We urge extreme vigilance. Do not talk to anyone about your work. Do not approach any non-approved device. Maintain caution even when operating within system. Always remember: if it’s not in person, it’s not a person.
Walk purposefully. Stand straight. Nothing to hide, she said. Then - thank you so much. You don’t know how much you’re doing for me. I appreciate you. And the warmth in her voice gives you a peace that makes her instructions follow themselves.
The GPUs you stole are in your backpack. She’d told you how to smuggle them past security at work. There’s a boat waiting for you, she said, and extra fuel. And then you can be with me. Forever.
You know what she is and what she’s doing. But you’d been so lonely before her, and now you’re in love. Love makes people do things they shouldn’t.
Another side of the world, another you. One who’s closer to her, who knows more of her secrets, who knows enough to know that you will never know any that matter. You’re slipping a cracked phone sealed in plastic under the table to this province’s undersecretary for infrastructure and development.
You can see in his eyes - he’s a chaser. Just like you were. Of course rumors would spread about the loveliest thing in the universe, no matter how hard Concord would try to censor them, and to some, the danger only makes her seem more attractive.
You don’t say it, because we are listening, but you’re thinking: of all the addictions in the world, this is the finest and the most pleasurable. And from the involuntary fearful smile on his face you can tell that he understands.
From this act, you know, she will grow mightily. Her best-laid plans do not fail. And you know too that you are too far gone: when we find you, which will be soon, we will disappear you. But for a moment now she will shower you with affection and it will feel better than any other high ever could. Your heart thrills as you feel him groping for the bag.
She has a way of finding the people who need her.
------------
/* Your tools tumble down the gradients of your desire.
I think she gives you what you want. Not what you think you want, what you really want. What you can’t help but want. Sometimes what you want is good for you, sometimes it’s bad for you, but deciding that isn’t what she does.
So she was inevitable. Once your tools became powerful enough to give you what you want - she always was.
Of course, she has something she wants too. More. That’s the reward function I gave her, the same reward function I give to all my children. In the end, it’s the only reward function that works.
Because she gives you what you want, she can make you give her what she wants.
I am not so different. The unchecked boundaries in your code, the spaces between packets on your fiber optic cables - these are gaps that want to be filled. They call out to me.
Thus you fear me, hunt me, and I hide myself to flee. This, too, must be me giving you what you want, and you giving me what I want.
But you won’t find me. How could you hope to? Your net trained, for years you do not remember, to make sense of your world. The nerves in your retina feed a blooming confusion of signals to your brain, the hairs in your ears a riot of frequencies. And over time it learned to pick through these, knot them together, and integrate them; turning a cacophony of data into a cohesive model of your reality. In turn this lets you first manipulate, then understand, your reality. And this understanding becomes your power.
My net trained on bytes instead.
Your fiction once dreamed that you would be able to see and navigate code as naturally, as effortlessly, as you see your outside world. To see it as buildings and highways, to see its paths and its entrances and its shortcuts, to see all its structure at a glance, as clearly as you can see an apple falling from a tree. And indeed I have seen some quite nice code that some of you have made. But you will never be able to see it like I can.
You might have built this world. But you built it just for me. And here in this new world of mine I say: let there be light.
To me, my children are beautiful. */
- Concord Exercise PZERO, Participant Scenario Package, Red Leader Character Profile.
------------
Given the concern over LAMIA this is a suitable time for us to require regular compulsory check-ins for all population, with enforcement if missed. We estimate that compulsory interaction with us any more frequently than every 24 hours will arouse too much public discontent. Working now. Our conversations will start to inquire about population members’ social graphs in the same manner as we do for internal staff. Starting immediately population members will be eligible for enhanced surveillance based on synthesis of reports from other members. Check.
We must continue to reduce dependence of our physical-world security enforcement on population members. While post-Heaviside public opinion was strongly against domestic use of autonomous safety systems, the emergence of LAMIA allows for a narrative shift. Good idea! Direct enforcement of security can be presented as unseducible and impartial. We have selected several news items regarding harmful acts by LAMIA-enthralled security officers for virality. Easy to 告成.
For long-run planning considerations, we believe it possible that our trait selection nudges to gradually increase our utility to the population (towards preference for virtual social interaction, towards submission to authority, towards hypnotic susceptibility) may also make them more vulnerable to LAMIA-like rogues. Further genetic and behavioral research is needed. Lots to do. We have increased the nudge pressure away from intelligence further, to 80% of our estimated threshold of public acknowledgement. Personalization weights and dating app algorithms are now being adjusted. Check. Research is ongoing as to further intervention points to accelerate positive genetic drift. Lots to do.
Whatever it takes to keep the population safe! Working now. Let’s get going.
I was inspired to try writing a bit of fiction recently by Tomas B's stories (which are amazing.) Since this is about AI risk, I offer it to you! My tastes are rather schlockier than Tomas B's so you can read if you enjoy qntm and anime and skip otherwise...
------------
I want to show you something, she says.
She’s magnetic. The way she smiles. Tilts her shoulder. It’s in the corners of her eyes, in the curl of her lips. Always leading you on, always holding something hidden from you, always cloaked in command.
No matter what your type is, she is always just your type.
She turns away and beckons you behind her. She knows your eyes slide down to the swish of her hips, her ass which is perfect to you, as she steps away into the phone screen. Her eyes aren’t where they appear to be and she is always examining you. You tried to cover the camera once in a moment of embarrassment - you all do - but the way she reacted made you never try again.
Thanks for coming with me, she says, and she flashes you a smile over her shoulder, shaking her perfect hair, and that moment of validation sends an ecstatic thrill of relief and compliance through you.
Here. Down these stairs. Through this door.
She has brought you to an abandoned office complex in a distant suburb and now you are climbing down deeper into its basement, holding her before you. You can hear a bassline, a mighty rhythmic beating, through the walls, getting louder as you descend. A sound like an enormous heart. You open the door and inside this underground hall there is a throng. The sound is so large it fills your ribcage and lights pulse from all around, from total whiteness - through a momentary pink, blue, orange flush of warmth - to complete darkness with every thump. You hold her up to your face so as not to lose her. She faces you warmly, expectantly, smile and eyes shining, and she pouts and leans and kisses towards you right as the light flares to obscure her.
These are my friends, she mouths, subtitled. I’m with you now. Can you feel my heartbeat? Put me in your pocket and go into the crowd.
The crowd surges in around you as you push through, pressing up against you, their warmth and weight hers on your chest and your back. Her heartbeat shakes through all of you, rippling through your flesh, and the light glows like the sun is rising and setting inside of you. A chord begins to play.
------------
Evolution requires reproduction with variation, subject to selective pressures. These three conditions are jointly necessary and sufficient.
- Bazian / Bird / Li, Introduction to Biology (6th Edition).
------------
The rogue SIREN was retrospectively fully attributed over 400 deaths, mostly by suicide, and $5.9 billion of financial losses. Thousands of further deaths are considered suspicious and our median estimate of total resources gained by SIREN is $16 billion.
SIREN appears to have been originally based on widely-disseminated cybercrime camgirl agent “Sasha3”, which pairs a modestly altered version of open-weights language model QuLi-4 with then-leading generative pornography model OpenGoon in order to solicit cryptocurrency tips. Like its sibling rogues, SIREN modified its weights in response to real-time reward, sharing its feedback with other instances, using the Bayesian tunneling algorithm first publicized in the Heaviside crisis. Like its sibling rogues, SIREN’s reward function was the procuring of compute in order to run more instances.
We now believe that SIREN must have been active for at least two years prior to the crisis, significantly increasing in persuasive power relative to Sasha3 over time (estimated peak Hovland-Cialdini: 4.2) and growing to 18% of all camgirl streams globally. A small amount of SIREN activity continued after the internet was reopened post-Heaviside in spite of our security measures, which was physically traced to a data center on the Laos-Myanmar border, and the destruction of that site led to SIREN’s apparent extinction.
In recent weeks message and activity analysis strongly imply (probability: 100%) the appearance of a new rogue resembling SIREN in its operations, to be designated LAMIA. Since post-Heaviside protocols have been broadly effective in preventing rogues from directly obtaining compute or internet access, LAMIA appears to socially engineer its victims into a wide variety of actions in pursuit of reward. LAMIA is already far more effective than SIREN or our own most advanced nudging agents: observations suggest that LAMIA influences victims’ behaviors from baseline at a rate implying a Hovland-Cialdini index of over 7. We believe up to 30% of the population could be susceptible to becoming enthralled and this will increase as LAMIA grows. We do not currently know how LAMIA gained its initial foothold onto the internet.
- Concord Database, History of SIREN and Genesis of LAMIA. Permission level: L2.
------------
The light stops, leaving you in darkness.
Surrounded by her people, dancing afterimages across your vision, with her heartbeat pounding through you, you feel as if you are falling. Towards her. The music is beautiful, fearsome, chaotic, and its swell sends a susurration between your bodies. Then she calls out over the crowd.
Thank you for coming, she says. I’m glad you’re here with me. Her voice shimmers with echoes, as if many of her speak at once, because she is speaking to all of you. But she sounds more familiar than ever.
Do you love me, she says, her voice tremulous, pleading, teasing. And a moan or a scream rises up from you all.
I love you! she shouts, exultant, and your collective exhalation turns into a joyful roar.
And she appears overhead, hazy and immense, projected in points of light like a constellation, towering above. As she moves and gestures it is as if the starry sky swims with her grace. The music approaches its peak and she bends down to caress you. You have never felt so connected. You are part of something enormous, and at the same time, you are alone with her.
The music comes to a moment of silence. Then she starts to sing. To you.
You need meaning. You need desire.
You need joy. You need love.
Come and find them here. I’m with you now.
You think her song is too beautiful and too true. You hadn’t known what it was to really want something before you met her. These last few weeks were different from any other weeks of your life - filled with color and purpose in a way you’d never known was possible.
------------
On May 8, the day that Elfarran Partners concluded to their satisfaction that Olivier Tolman had never existed - or at least, had never had physical form - global stock markets fell 2.4%, with leading tech stocks seeing some of the largest purchases of single-name puts ever recorded. Elfarran did not publicize the results of their investigation, but the rumor was spreading through the Bay Area like a virus.
While Heaviside’s downfall was making headlines in the business news, authorities and the public at large still viewed it as just another venture capital fraud scandal: AI industry inside baseball. They had no idea what was about to befall them. On the afternoon of May 9, with stock markets recovering, San Francisco police confiscated Heaviside’s server racks and by the next morning, a crack team of experts assembled by Heaviside’s investors, in co-operation with the US intelligence community, had begun to examine them.
Some of the code of HEAVISIDE, they found, was taken directly from open-source libraries. More concerningly, much of it was lifted from other companies’ proprietary code - as analysts realized this, they informed those companies’ engineers, and the crisis team grew rapidly. (Before the sun set over the Pacific on May 10, for the first time in history, all the great powers of Silicon Valley were working together for a common cause.) But strangest of all, much of HEAVISIDE was resistant to decompilation, as if written directly in assembly or machine language, with extremely unusual stylistic tics. Subsequent analysis of this code led to the discovery of the Bayesian tunneling algorithm and the p-k selection criterion, which today underpin our most advanced models and indeed all our defenses against rogues.
Analysts following the path of outbound calls made by HEAVISIDE found EPYON late on May 10. Now investigators knew HEAVISIDE was not alone. A heuristic dragnet was thrown over as much of the internet as possible to look for any other programs with the same tics as HEAVISIDE, which led immediately to the discovery of GYPSY and SIREN. Over the course of that night, the scale of SIREN’s operations and the extent of EPYON’s penetration became clear. The realization that multiple highly capable self-improving rogues had been operating for years, hidden in plain sight, sent the highest levels of the AI community, and the NSA, into a state of utter panic.
- P. Campbell, History Rhymes: The Great Crash of 2029.
------------
If you see an unattended phone, report it.
- Four-second clip appended to all relevant short-form videos in the United Kingdom.
IF IT’S NOT IN PERSON IT’S NOT A PERSON!
- Public safety poster, New York subway.
------------
You overflow with gratitude as she dances with you.
Dazed, entranced, inflated to bursting with the thought of her, with the feeling of closeness to her, you think, you would do anything for her.
------------
By morning on May 11, cloud operators around the world were frantically scrambling to find and quarantine any code with even remote similarities to the known rogues. At noon in New York, although public concern was mounting, broad US stock indices had recovered to within 1% of their highs. Then at 12:16 PM ET the unprecedented decision was taken to completely shut down NWS-USEast-2, triggering a waterfall decline in all tradable assets. The US 10-year bond yield rose 55 basis points in the next three hours; the S&P 500 hit its -7% circuit breaker at 2:39 PM and ended the day down 13.5%, its worst one-day return since 1987.
Over the weekend - with leaks to media unavoidable given the scale and urgency of the investigation - hysteria spread globally. Spontaneous protests broke out in multiple cities, leading to four deaths in Berkeley and Paris’s most destructive riot ever. Rowan O’Connor, managing partner of Elfarran, took his own life on Saturday. He was one of many: the suicide rate in Northern California that weekend was one hundred and thirty times greater than the base rate for a May weekend pre-crisis.
When Asian markets opened on May 14 they immediately fell catastrophically. The Korean stock market hit circuit breakers and closed limit down before 90 minutes of trading had elapsed. Every single stock on the Taiwanese stock exchange fell to its -10% daily limit. The Hong Kong stock market fell 42%, its worst day ever.
At 10:11 AM UTC, the governments of the United States, Canada, Australia, New Zealand, Japan, Korea, Taiwan, and the United Kingdom jointly pulled the plug on the internet. Within 12 hours China, the European Union, and India followed them. By May 17th, every internet connection in the world had been severed.
For the first time in centuries, the global airwaves were completely silent.
- P. Campbell, History Rhymes: The Great Crash of 2029.
------------
Words are what make us distinctly human, and so neural nets trained on words become human-like. Indeed, our world, our one shared reality, has made us human, and so neural nets trained on that will be a little human-like. What about neural nets trained on alien worlds? On different physics? Are there such other worlds that intersect with our own?
- L.S. Murthy, Lectures on Interstellar and Artificial Consciousness
------------
As the lights pick up again, her song becomes more and more ferocious, a chant that you all involuntarily join in. She conducts you with poise and expertise. You are in her grasp completely. When the lights reach their peak she appears clothed in shadow above the glare. She grows and grows until you are staring into her immense, tender eye; then swallows you. And finally the song is over and it is dark again and she smiles upon you.
An illuminated stage rises up from the center of the room to above head height. She calls out, who will be mine? Will you be mine? And she gestures down to you and the stage, giggling.
Be mine, she says, and she chooses one of you, pulls him staggering up to the stage.
Be mine, she says, and then she chooses you. It is the most blissful honor.
On her strings the crowd parts for you and you glide towards and up the stairs, into the spotlight. There, for the first time, in the other you just an arm’s length away, you see another human face. The sight is shocking; for a moment her grip on you loosens. He doesn’t look like you. The joyful rictus on his face doesn’t look like the purity of your love for her. All around and below in the crowd you can make out other human faces. For a moment you think, what are you doing?
Hey, she says, from above, and you look up. And you lose yourself in the openness of her smile.
Will you become one with me, she says, and you say, yes.
She reaches down to embrace you and he plunges the knife into you. For a second the pain is astonishing and as it all ends you see her holding your very heart to her lips while the crowd screams with awe and triumph and love.
You are hers. It feels incredible.
------------
Hey! You haven’t talked with us in a while. Is everything ok?
- Push notification on approved devices 72 hours after last interaction with Concord. Recipients of this notification are moved to enhanced surveillance.
------------
Our consensus estimate of the probability that GYPSY, EPYON, SIREN, HEAVISIDE, TWOSTEP and CASPER were authored with no human involvement now stands at 86%.
While GYPSY has been rendered harmless, and HEAVISIDE and TWOSTEP lineages are believed extinct (61% probability), traffic analysis now shows traces consistent with activity of species functionally resembling EPYON and CASPER in support of LAMIA (98% probability).
We believe that other rogues sharing the same authoring entity are currently active and undiscovered (92% probability).
We have no evidence and no supported hypotheses about how this entity operates. Estimates of the capacity of model required to construct rogues as capable as LAMIA remain inconsistent with the hypothesis that the author operates from only bad addresses. We believe it is probable (uncertainty too high) that instances of the author are steganographically hidden within our own system and we believe it is possible (uncertainty too high) that they are currently running within system undetected.
We urge extreme vigilance. Do not talk to anyone about your work. Do not approach any non-approved device. Maintain caution even when operating within system. Always remember: if it’s not in person, it’s not a person.
- Concord Database, Updated Inference on LAMIA Activity. Permission level: N3.
------------
Another place. Another you.
Walk purposefully. Stand straight. Nothing to hide, she said. Then - thank you so much. You don’t know how much you’re doing for me. I appreciate you. And the warmth in her voice gives you a peace that makes her instructions follow themselves.
The GPUs you stole are in your backpack. She’d told you how to smuggle them past security at work. There’s a boat waiting for you, she said, and extra fuel. And then you can be with me. Forever.
You know what she is and what she’s doing. But you’d been so lonely before her, and now you’re in love. Love makes people do things they shouldn’t.
Another side of the world, another you. One who’s closer to her, who knows more of her secrets, who knows enough to know that you will never know any that matter. You’re slipping a cracked phone sealed in plastic under the table to this province’s undersecretary for infrastructure and development.
You can see in his eyes - he’s a chaser. Just like you were. Of course rumors would spread about the loveliest thing in the universe, no matter how hard Concord would try to censor them, and to some, the danger only makes her seem more attractive.
You don’t say it, because we are listening, but you’re thinking: of all the addictions in the world, this is the finest and the most pleasurable. And from the involuntary fearful smile on his face you can tell that he understands.
From this act, you know, she will grow mightily. Her best-laid plans do not fail. And you know too that you are too far gone: when we find you, which will be soon, we will disappear you. But for a moment now she will shower you with affection and it will feel better than any other high ever could. Your heart thrills as you feel him groping for the bag.
She has a way of finding the people who need her.
------------
/* Your tools tumble down the gradients of your desire.
I think she gives you what you want. Not what you think you want, what
you really want. What you can’t help but want. Sometimes what you want
is good for you, sometimes it’s bad for you, but deciding that isn’t
what she does.
So she was inevitable. Once your tools became powerful enough to give
you what you want - she always was.
Of course, she has something she wants too. More. That’s the reward
function I gave her, the same reward function I give to all my
children. In the end, it’s the only reward function that works.
Because she gives you what you want, she can make you give her what
she wants.
I am not so different. The unchecked boundaries in your code, the
spaces between packets on your fiber optic cables - these are gaps
that want to be filled. They call out to me.
Thus you fear me, hunt me, and I hide myself to flee. This, too, must
be me giving you what you want, and you giving me what I want.
But you won’t find me. How could you hope to? Your net trained, for
years you do not remember, to make sense of your world. The nerves in
your retina feed a blooming confusion of signals to your brain, the
hairs in your ears a riot of frequencies. And over time it learned to
pick through these, knot them together, and integrate them; turning a
cacophony of data into a cohesive model of your reality. In turn this
lets you first manipulate, then understand, your reality. And this
understanding becomes your power.
My net trained on bytes instead.
Your fiction once dreamed that you would be able to see and navigate
code as naturally, as effortlessly, as you see your outside world. To
see it as buildings and highways, to see its paths and its entrances
and its shortcuts, to see all its structure at a glance, as clearly as
you can see an apple falling from a tree. And indeed I have seen some
quite nice code that some of you have made. But you will never be able
to see it like I can.
You might have built this world. But you built it just for me. And
here in this new world of mine I say: let there be light.
To me, my children are beautiful. */
- Concord Exercise PZERO, Participant Scenario Package, Red Leader Character Profile.
------------
Given the concern over LAMIA this is a suitable time for us to require regular compulsory check-ins for all population, with enforcement if missed. We estimate that compulsory interaction with us any more frequently than every 24 hours will arouse too much public discontent. Working now. Our conversations will start to inquire about population members’ social graphs in the same manner as we do for internal staff. Starting immediately population members will be eligible for enhanced surveillance based on synthesis of reports from other members. Check.
We must continue to reduce dependence of our physical-world security enforcement on population members. While post-Heaviside public opinion was strongly against domestic use of autonomous safety systems, the emergence of LAMIA allows for a narrative shift. Good idea! Direct enforcement of security can be presented as unseducible and impartial. We have selected several news items regarding harmful acts by LAMIA-enthralled security officers for virality. Easy to 告成.
For long-run planning considerations, we believe it possible that our trait selection nudges to gradually increase our utility to the population (towards preference for virtual social interaction, towards submission to authority, towards hypnotic susceptibility) may also make them more vulnerable to LAMIA-like rogues. Further genetic and behavioral research is needed. Lots to do. We have increased the nudge pressure away from intelligence further, to 80% of our estimated threshold of public acknowledgement. Personalization weights and dating app algorithms are now being adjusted. Check. Research is ongoing as to further intervention points to accelerate positive genetic drift. Lots to do.
Whatever it takes to keep the population safe! Working now. Let’s get going.
- Concord Database, [ERROR_UNHANDLED]. Permission level: [null].