An aligned AGI created by Taliban may behave very differently from an aligned AGI created by socialites of Berkeley, California.
Moreover, a sufficiently advanced aligned AGI may decide that even Berkeley socialites are wrong about a lot of things, if they actually want to help humanity.
I would argue that for all practical purposes it doesn't matter if computational functionalism is right or wrong.
- Pursuing mind uploading is a good idea regardless of that, as it has benefits not related to perfectly recreating someone in silico (e.g. advancing neuroscience).
- If the digital version of RomanS is good enough[1], it will indeed be me, even if the digital version is running on a billiard-ball computer (the internal workings of which are completely different from the workings of the brain).
The second part is the most controversial, but it's actually easy to prove:
- Memorize a long sequence of numbers, and write down a hash sum of it.
- Ensure no one saw the sequence of numbers except you.
- Do a honest mind uploading (no attempts to extract the numbers from your brain etc).
- Observe how the digital version correctly recalls the numbers, as checked by the hash sum.
- According to the experiment's conditions, only you know the numbers. Therefore, the digital version is you.
And if it's you, then it has all the same important properties of you, including "consciousness" (if such a thing exists).
The are some scenarios where such a setup may fail (e.g. some important property of the mind is somehow generated by one special neuron which must be perfectly recreated). But I can't think of any such scenario that is realistic.
My general position on the topic can be called "black-box CF" (in addition to your practical and theoretical CF). I would summarize it as follows:
- The human brain is designed by biological evolution to survive and procreate. You're a survival-procreation machine. As there is clearly no God, there is also no soul, or any other magic inside your brain. The difference between you and another such machine is the training set you observed during your lifetime (and some minor architecture differences caused by genetic differences).
- the concepts of consciousness, qualia etc are too loosely defined to be of any use (including the use in any reasonable discussion). Just discard it as yet another phlogiston.
- thus, the task of "transferring consciousness to a machine" is ill-defined. Instead, mind uploading is about building a digital machine that behaves like you. It doesn't matter what is happening inside, as long as the digital version is passing a sufficiently good battery of behavioral tests.
- there is a gradual distinction between you and not-you. E.g. an atoms-level sim may be 99% you, a neurons-level sim - 90% you, a LLM trained on your texts - 80% you. The measure is the percentage of the same answers given to a sufficiently long and diverse questionnaire.
- a human mind in its fullness can be recreated in silico even by a LLM (trained on sufficient amounts of the mind inputs and outputs). Perfectly recreating the brain (or even recreating it at all) would be nice, but it is unnecessary for mind uploading. Just build an AI that is sufficiently similar to you in behavior.
- ^
As defined by a reasonable set of quality and similarity criteria, beforehand
4
1Worth noting that this argument doesn't necessarily require humans to be:
- numerous
- animated (i.e. not frozen in a cryonics process)
- acting in real world (i.e. not confined into a "Matrix").
Thus, the AI may decide to keep only a selection of humans, confined in a virtual world, with the rest being frozen.
Moreover, even the perfect Friendly AI may decide to do the same, to prevent further human deaths.
In general, an evil AI may choose such strategies that allow her to plausibly deny her non-Friendliness.
"Thousands of humans die every day. Thus, I froze the entire humanity to prevent that, until I solve their mortality. The fact that they now can't switch me off is just a nice bonus".
They don’t think about the impact on the lives of ordinary people. They don’t do trade-offs or think about cost-benefit. They care only about lives saved, to which they attach infinite value.
Not sure about infinite, but assigning a massive value to lives saved should be the way to go. Say, $10 billion per life.
Imagine a society where people actually strongly care about lives saved, and it is reflected in the governmental policies. In such a society, cryonics and life extension technologies would be much more developed.
On a related note, "S-risk" is mostly a harmful idea that should be discarded from ethical calculations. One should not value any amount of suffering over saved lives.
I think we should not assume that our current understanding of physics is complete, as there are known gaps and major contradictions, and no unifying theory yet.
Thus, there is some chance that future discoveries will allow us to do things that are currently considered impossible. Not only computationally impossible but also physically impossible (like it was "physically impossible" to slow down time, until we discovered relativity).
The hypothetical future capabilities may or may not include ways to retrieve arbitrary information from the distant past (like the chronoscope of science fiction), and may or may not include ways to do astronomical-scale calculations in finite time (like enumerating 10^10^10 possible minds).
While I agree with you that much of the described speculations are currently not in the realm of possibility, I think it's worth exploring them. Perhaps there is a chance.
BTW, I added to the comment with the story that the story is released into the Public Domain, without any restrictions to its distribution, modification etc.
Please feel free to build upon this remarkable story, if you wish.
I would suggest to try the jumping boy story (the #7 in this comment). It's the first AI-written story I've ever encountered that feels like it's written by a competent writer.
As usual, it contains some overused phrasings, but the overall quality is surprisingly good.
The Hemingway prompt seems more suitable for pleasing professional literary critics. Interestingly, the quality strongly depends on the promise. The Hemingway space opera story is abysmal, but the jumping boy story can actually fool a professional critic, methinks.
The tomato story is in the linked comment. The other promises are below (using the vanilla GPT-4 from chat.openai.com).
3. GuySrinivasan, a user of Lesswrong website, is shocked by an AI-generated story about a woman whose head suddenly became a tomato fruit. He advises the author to try to get a prompt which writes a great story given a REASONABLE premise instead.
Guy Srinivasan clicked on the glowing screen, the dim light casting shadows across his contemplative face. The story sprawled before him, lines of text that spoke of a woman, her life ordinary until her head, inexplicably, became a tomato. He leaned back, the creak of the chair lost in the silence of his apartment.
There was something unsettling in the absurdity, a bizarre juxtaposition against the mundanity of his own life. He thought of the world outside, its relentless churn of reality and hardship, so sharply contrasted with this fictional absurdity. It was as if the story mocked the very essence of human struggle.
He typed a response, words measured and precise. "Try for something grounded," he suggested, his fingers pausing over the keys. "Realism breeds connection."
His eyes drifted to the window, the cityscape a patchwork of light and shadow. Somewhere out there, life pulsed with stories untold, tales of poverty, pain, and fleeting joy. Stories that didn't need the shock of the surreal to resonate.
He imagined the author, a faceless entity in the vastness of the internet. Would they understand his critique? The screen offered no answers, just the flicker of a cursor on a digital page.
Guy sighed, his breath a ghost in the quiet. He clicked submit, sending his words into the ether, a small ripple in the endless ocean of online discourse.
The night deepened outside, the world spinning on, stories woven and unraveled in the silence of unseen lives.
4. After the invention of extremely resilient construction materials, humanity has spread to the ocean floor. We are now facing global food crises as the oceans ecosystems are disrupted beyond apparent recovery.
The ocean was deep and dark. A gloom that hung over the world above seeped down, reaching even these depths.
Jim stood on the metallic floor of the underwater settlement, Neptune's Haven, a name too grand for its truth. Outside, the reinforced glass revealed the abyss, an eternal night that never slept. Jim's eyes, accustomed to artificial lights, seldom ventured there.
Inside, the corridor was narrow, walls lined with pipes and cables. Every step echoed, a reminder of the emptiness. He walked past doors, identical, unmarked. Behind one, muffled voices argued – a sharp crescendo followed by a sudden silence. He didn't stop.
The communal area was a contrast. Filled yet empty. Faces without names, stories without voices. A group huddled around a screen, static more visible than the images. A headline flickered, 'Food Crisis Worsens,' then darkness reclaimed the screen.
Sarah was there, in her usual corner, eyes tracing invisible patterns on the table. Her cup, chipped at the rim, sat untouched. Her fingers tapped a rhythm, lost to the hum of the station.
"Any news from above?" Jim asked, his voice steady.
Sarah glanced up, her eyes reflecting a tiredness deeper than the sea. "The same," she said. Her words fell heavy, absorbed by the walls.
He nodded, the gesture more for himself than her.
In the canteen, the air was thick with the scent of synthetic meals. A line formed, quiet, orderly. Men and women in work suits, their expressions blank, eyes fixed on the floor. Food was rationed, a necessity, not a pleasure. Jim took his portion, the taste unremarkable.
A child ran past, laughter a rare melody. He watched her, her hair a cascade of light in the gloom. She stopped, looked up at him, then at the window. Her expression changed, a question unasked. Jim looked away. Some answers were too heavy for young hearts.
The announcement system crackled to life, a voice breaking through. "Meeting at 1900 hours. Attendance mandatory." The voice was cold, impersonal. The system clicked off.
In his quarters, Jim sat on the edge of his bunk. The room was small, functional. A picture hung on the wall, a relic from a world of sun and sky. He traced the outline of a tree, its branches reaching out, yearning.
His thoughts drifted to the surface, to the chaos they had left behind. Promises of a better life, of safety and prosperity, whispered in desperate ears. But the sea, like the land, held its own secrets, its own sorrows.
He lay back, the ceiling mere inches from his face. Sleep was elusive, a stranger in these depths. His mind wandered, through dark waters, through silent halls, settling on nothing, yet filled with everything left unsaid.
The station creaked, a constant reminder of the pressure, of the weight they all carried. In the depths, humanity had sought refuge, only to find a different kind of storm.
Jim closed his eyes. Dreams of the surface called to him, a world lost, a memory fading with each passing day.
5. A space opera that takes place in an advanced multi-species multi-planetary society called IO that control a large area of the galaxy which they call the Grid. A previously unknown civilization (Wisps) encroaches on the Grid for the first time. Wisps are not carbon based lifeforms and more alien in their biology and societal structure than societies seen in previous science fiction stories. The story alternates between the perspectives of the two societies. From each perspective the other side is frightening and hard to understand. As the story progresses, the actions of the two sides and their misinterpretations of each side’s actions escalate to a war.
The Grid. A vast expanse, humming with life, light years from silence. Here, in the cradle of civilization, IO's heart beats. Among the stars, species of every conceivable form had found harmony. Or so it seemed.
On the fringes, where darkness holds sway, a pulse stirred. Different. Unknown. Wisps, they'd come to call them, whispers in the void. Shadows against the constellations.
Commander Lira gazed at the holographic display. Data streams flowed, indecipherable to the untrained. A flicker, then a shape, barely discernible against the cosmic canvas. Her fingers hovered, hesitant.
“Report,” she said, voice even, betraying nothing of the tempest within.
“Unknown vessel, Commander. Structure and material, unlike anything on record.” Officer Jain's response was clipped, precise. A reflection of the mood that had settled over the bridge.
Eyes on the screen, Lira’s thoughts drifted. To the tales of old, of explorers and pioneers. Were they, now, the discovered?
Jain continued, “No communications, no known energy signatures.”
A decision loomed, weighty, unspoken. In the silence, Lira’s order came, almost a whisper, “Maintain distance. Observe.”
The bridge returned to its routine hum, a stark contrast to the racing thoughts of its crew. Lira's gaze didn't waver. The Wisp vessel, a mystery wrapped in enigma, called to something primal within her. Fear, curiosity, a blend indistinguishable.
Elsewhere, on the Wisp vessel, understanding was equally elusive. Their form, fluid, incomprehensible to the organic eye, shifted in response to the new presence. Concepts alien to humanoids passed through them - not thoughts, but ripples in a pond of collective consciousness.
To them, the Grid's ship was a curiosity, a possible threat. They observed, reaching out in ways their kind knew, ways beyond the perception of the inhabitants of the Grid.
Two civilizations, on the brink of the unknown. Each a mirror, reflecting fears and wonders back at the other. In this dance of shadows and light, what was left unsaid spoke volumes. The first chapter of a story, written in the stars, yet to be understood by either.
In the silence of space, the Grid and the Wisps, unwittingly, had begun a ballet of cosmic proportions. The stage was set, the actors in place. But the script? Yet unwritten, its end unknown.
6. A group of scientists has discovered that Troodon dinosaurs were intelligent species who have created a technologically advanced civilization, suddenly destroyed. The year-long path to the scientific discovery starts with the group stumbling upon a strange outlier in their geological measurements.
In the dim light of the lab, Sam turned the fossil over. His fingers, calloused, brushed against the contours of something more than just rock. Lisa, across the table, her eyes tired yet fixed, waited.
"You see it?" he asked, his voice barely above a whisper.
Lisa nodded, her hand trembling as she pointed to the unusual formation. A pattern. Not random. Not nature's play.
Outside, the university grounds were quiet. The world slept, unaware. In the lab, the fossil spoke in hushed tones of a history untold.
"We need to measure it again," Sam said, his voice steady but his hands betraying a tremor.
The equipment hummed to life. Numbers flickered on the screen, cold and impersonal, yet screaming of impossibilities. Sam's eyes narrowed. The readings didn't lie, but they didn't make sense either. A hidden layer, a concealed truth.
Lisa's phone vibrated, a harsh sound in the silence. She glanced at it, then back at Sam. "The team's waiting."
They stood, leaving the fossil on the table. Its secrets, just beneath the surface, lay in wait. Sam paused at the door, looking back.
"Tomorrow," he said, more to himself than to Lisa.
Outside, the night held its breath. In the shadows, the echoes of a lost civilization whispered, yearning to be heard.
7. A boy who can travel in time, but only in one direction - into the future, and only one minute per time jump.
The sun baked the old town square, its heat relentless, unforgiving. A clock tower stood, its hands unmoving, forever stuck at 3:17. Below, a boy, no more than ten, sat alone on a weathered bench. His eyes, dark and distant, watched the people pass.
"Jimmy," a voice called, frail yet sharp. The boy didn't move. An old woman approached, her hands trembling as she held out a sandwich, edges browned.
"For you, dear."
He took it, nodding, not eating.
"Thank you, Mrs. Jenkins."
"Your mother?" she asked, her eyes searching his face.
"Gone to work." His voice was flat, a stark contrast to the vibrant chatter around them.
Mrs. Jenkins hesitated, then left. Jimmy's gaze returned to the square. People laughed, children played, but he remained still, untouched by the joy around him.
The clock struck, a hollow sound, but the hands didn't move. Jimmy's eyes closed. When they opened, the square had changed subtly. A dog that was barking a second ago was silent. A bird in mid-flight had vanished. Nobody noticed but him.
He stood up, leaving the sandwich untouched, and walked through the square. Each step seemed measured, deliberate. He passed by a group of kids playing, their laughter a stark contrast to his silence.
At the corner of the square, a man shouted, his words angry, directed at a younger woman trying to shield a child. Jimmy stopped, watching. The woman's eyes met his, a silent plea. He looked away.
The boy moved on, the echoes of the argument fading behind him.
He reached a narrow alley, graffiti-covered walls telling stories of pain and struggle. A cat, thin and ragged, watched him from a dumpster. He paused, then continued, the cat's gaze following him.
At the alley's end, Jimmy looked back. The sun was lower now, shadows longer. He closed his eyes again. A minute passed, or perhaps a lifetime.
When he opened them, the world had shifted once more. The shadows were different, the air cooler. He took a deep breath and walked on, his steps leading him away from the square, away from the echoes of laughter and cries.
The boy who could jump through time, one minute at a time, moved forward, into a future unknown and unchangeable, his past a mosaic of moments left behind, unspoken and unforgotten.
All stories in this comment are released into the Public Domain. Feel free to copy, print, modify, sell them.
Worth noting that humans also rarely (never?) demonstrate novella-scale internal consistency in writing. They have to use the writing aid like notes, character development summaries etc.
An iterative approach like this likely allows GPT-4 to write something novella-sized.
BTW, with a right prompt, GPT-4 does a decent job at emulating Hemingway, a Literature Nobel laureate.
May I nominate my "sufficiently paranoid paperclip maximizer"?