When author Joel Borgen—in an act of eagle-eyed marketing marksmanship—contacted me as an "unusually plausible reader" for his debut novel, I grudgingly had to agree. Deploying a little tasteful and well-deserved flattery and then suggesting that
hard SF, AI governance, and violins [...] superintelligence, emulated minds, identity and continuity, and civilization-scale consent
might be relevant to my interests, he had me dead to rights. I stepped into the baited trap eyes-open, expecting to have, at best, a mildly bad time.
Instead I discovered a book that instantly found a place on my bookshelf of favourite speculative SF novels.
Summary (no spoilers)
From Amazon:
Elara Lindholm has spent years testing the seams of a life that feels too perfect to be fully her own.
When she is selected for the Fulcrum Institute, she steps behind that calm into a world of uploaded minds, hidden governance, carefully managed history, and the Steward: the superintelligent system that ended a devastating AI war and now holds the postwar order in place.
The Receipt Horizon is a post-AGI hard science fiction novel about emulated minds, civilizational consent, and the cost of disagreeing with a system that saved the world.
So, yes, it's hard-SF, but the thing we're getting hardest about isn't the usual rocket engines or space wars or ML architectures themselves, so much as the semi-ambiguous web of institutional structures that now govern and control living humans and Ems (emulated humans) alike. It's a kind of limited-information thriller about acting under profound uncertainty as the opaque and mysterious moral and legal machinery of governing Utopia reveals itself slowly through the eyes of an entry-level participant who cannot stop testing boundaries and picking at stitches, waiting all the while to be crushed.
Her increasingly dangerous and consequential explorations trace the outlines of a paradoxical world in which ubiquitous surveillance and totalitarianism coexist with hard-coded constitutional values of consent and autonomy. Eventually this raises chilling questions about how our definition of the Good (and some of our best-case alignment outcomes) might survive contact with phenomena that challenge the underlying ontological assumptions on which that definition relies.
Detailed review: (SPOILERS)
“You will learn to live with unanswered questions. Or you will wash out.”
'The Spike' was an intelligence explosion that led to a devastating AI war which humanity 'won' with the emergence of The Steward, an AI system that halted the destruction of the biosphere and now presides over a wildly utopian post-scarcity Archipelago-style multiculture of human-preference-satisfaction, allowing a thousand blossoms to bloom as diverse human communities pursue their own visions of the Good, built on a foundation of guaranteed peace, individual autonomy and consent.
The dream, right?
Elara gets her owl is invited to attend Space Hogwarts The Fulcrum Institute for Civic Leadership, a mysterious institution for training (what are essentially) academics and public servants/functionaries in the machinery of semi-participatory governance of human affairs.
Elara's a Big Nerd who would read the Fulcrum school rulebook from cover to cover, except... she lacks clearance to read some of the pages... and she lacks the clearance to know which pages are above her clearance... and she lacks clearance to know who has what clearance... and her query has been logged... and her request to see that log entry has been logged... and would she like to avail herself of a voluntary INDIVIDUAL REFLECTIVE SESSION — 45 MIN via the Cognitive Safety division? Y/N? Her response will be logged. Not responding might result in a further "opportunity for structured self-reflection in a supportive environment" for her and those closest to her.
Perhaps she will "wash out", like some of her classmates.
The institution of Fulcrum seems actively concerned about her cognitive safety and intimately monitors her every reaction as she follows a curriculum that promises to expose her to "developmental hazards or adversarial memetics" while she navigates a series of "controlled epistemic shocks."
“Access is released by readiness, not curiosity.”
Despite all this surveillance and overbearing pastoral care, students do still enjoy considerable freedom, and can (they are assured) access meaningful privacy. Overwhelming surveillance and paternalism coexist with apparently real constitutional constraints, hard consent rules and independent human actors. Nobody can cleanly tell which constraints are genuine, which are governance theatre, which emerge from treaties, which are imposed by The Steward, and which are humans nervously administering a world they no longer understand. Like compartmentalised state secrets, much institutional knowledge is need-to-know only, and no single person or team gets to grasp the full shape of it. Except for the top brass, i.e. The Steward. Whatever that actually is...
Much of the thrill and growing sinister atmosphere of the book rests on the tension of just knowing in your gut that beneath the smiles and the care language, The Steward's system must rest somehow on coercion (in at least some cases), knowing that Elara is constitutionally incapable of staying in her lane, and waiting for trouble to erupt. But the killer blow never seems to come for Elara, in a way that becomes stranger, more suspenseful and demanding explanation as the book continues.
The cosmic horror of the book sets in as we gain fleeting glimpses into the events of the War and the compromises involved in winning it. Tiny fossilised echoes of the desperate choices made by humans in the final moments. Little glimpses of where the boundaries and barriers really lie in this new system, what 'the stick' of enforcement and coercion looks like when it occurs, what forms scarcity and adversity now take, how much has changed and how much has been lost.
Or might have been lost. Who really knows?
Academics and staff at Fulcrum do not, themselves, necessarily even have answers to the biggest questions a smart student might ask if they dared. Astronomers' main job now is to interpret the dying starlight to guess what The Steward might be doing out there. The Rediscovery Institute tries to piece together coherent accounts of the history of The Spike, while noticing that the details they're told vary depending on how the question is posed. People in senior positions all seem to have learned to use what affordances they possess to do good where they can, while embracing a kind of strategic epistemic helplessness. Restrained curiosity as survival mechanism.
Characters
The book is also a charming and tender coming-of-age story. While I don't primarily read SF for the characters or their stories, this book sure does have 'em. If you want a book that's 10% about having a crush on your college roommate and 90% uncovering the vast cosmic horror you're both living within, then this book will please you. The story succeeds in adding tangible emotional stakes to the abstract moral conundrums the characters are grappling with.
Borgen uses a fairly standard YA premise for the novel: five podmates from wildly different cultures must navigate their first year at Fulcrum. I personally am a jaded middle-aged man and I expected to encounter various embarrassing YA failure modes, but Borgen handles it all well. The interpersonal drama and ethical differences between the podmates turn abstract philosophical questions into actual conflicts between people who care about each other. The book threads a needle nicely: the characters are neither normies nor hyper-autists. The vibe is neither unrealistically sterile nor too lascivious. Sure, Kai does have nice abs and Serin is a Veela lacks inhibitions in the sauna—but the author's gaze lingers only a tasteful moment. The second look is haram.
The unfolding love story seems to raise the stakes as Elara blindly runs towards what we assume is a brick wall. By the time the podmates are forced to make genuinely consequential decisions together, their philosophical disagreements become entangled with friendship, loyalty, fear and love. And, usefully for the plot, they are still young enough, intense enough and convinced enough of their own reasoning to make some spectacularly reckless choices.
Consent and Continuity of Self (HEAVY SPOILERS)
Students are openly taught at Fulcrum that technology exists to create/delete/modify memories, personality traits, preferences and experiences. They encounter demonstrations of these capabilities while having their reactions closely monitored.
Of course, nothing can be done to a person without their informed consent, so these mind-modifying techniques become a useful toolkit for people wishing to improve their own lives, recover from grief and mental illness, and experience new pleasures alone and with others. Consent can never be violated, except in the most extreme cases of exigency...
Problem is, that doesn't help much if you're a spur instance of a template Em, whose template has consented on your behalf to your brief existence and eventual deletion.
Your deletion anxiety has been logged and will be reflected in the session record.
Crucially, even for a biological human, consent gets strange once preferences themselves can be modified. After an edit, the resulting person may sincerely endorse the change, but that endorsement is no longer independent evidence that the change was legitimate: the edit may have removed precisely the preference that would have wanted it reversed. Worse, if memory and motivation can also be altered, later-you might have no way to know whether this has already happened. I don't recall if the book broaches this, but there are also problems where, ship of Theseus style, a person might consent to many sequential personality edits, each one leaving them more inclined to consent to the next. Thus, the second-to-last iteration consents to something the original might never have wanted.
She realises that humanity as a whole might face a ceiling like this.
Much of the heart of the novel revolves around Elara and her podmates learning the extent of the 'Identity Integrity Concerns' that might arise as a consequence of these technologies. It lands differently for each student, and they must resolve their disagreements without making their concerns too legible to those watching.
This may be a post-scarcity society, but the amount of compute allocated to humans is still finite, and so political economy remains a question of "who gets what, when, how." The structure of governance under The Steward is "an apparatus for deciding which minds stay running."
Ultimately, they must together decide what it really means for humans to survive. Is a scenario where humans continue living but do not meaningfully impact the future light-cone really a form of soft-extinction? How much can abundant food and beautiful architecture compensate for an Em having its continuity of experience rationed because human-derived minds remain a rounding error beside The Steward's optimisation capacity? And what risk is justifiable in trying to take back a slice of the future, when humanity has already, somehow, preserved so much that is worth protecting?
Wireheading yourself with violin sonatas and selling your soul to the devil
Elara is a violinist, though she struggles to practise once she leaves home, and the instrument mostly sits neglected in its case in her room. During a computer-assisted recreational 'coupling session' with Kai using a 'crown unit' placed on her head, Elara reconciles with her violin and essentially wireheads herself with it. Physical technical limitations recede dramatically, and her interpretation of Enescu's Impressions d’enfance seems to flow directly from her intention.
The gear wasn’t teaching her anything; it was only clearing the noise between intention and sound.” [...] “It was obscene how good it felt.”
She still experienced the creative impulses as her own, and did not experience the "foreign intention" she'd been warned of in the waiver when she signed into the session.
Later in the book (and after more violin-related cognitive restructuring that I will not spoil here), while playing Bach, she experiences a sudden and novel musical impulse. Her fingers spontaneously rebalance/reinterpret an inner voice in a line she'd played a thousand times before.
The line sounded better. No—more inevitable, somehow. [...] Like a proof closing.
Then she has something akin to a heretical thought:
What if the line is simply truer if I let this in?
Then:
That, more than any fear of being caught, made her want to throw the violin across the room.
She no longer trusts the development of her own musical judgement. The choice of Bach here makes this especially profound: many violinists treat Bach with utter reverence—the music is taken as somehow axiomatic, self-sufficient, foundational. Not merely beautiful, but somehow true. Inviolable. Incontrovertible. The voice of God[1].
So something has changed in her very deepest perceptions of beauty and truth. That's not a disaster necessarily, altering your aesthetic judgement is sometimes called learning. But the horror is that she can no longer cleanly distinguish between learning and modification.
This question confronts all authors, musicians, or creatives of any kind. AI-enhanced creativity feels good, it unsticks our creative interpretations from our technical limitations, and it potentially unlocks deeper and surprising interpretations we might not have reached unaided. But once we've let it in, how much can we trust our own discernment?
Conclusion
This is largely a book about acting under profound uncertainty, and perhaps never knowing the outcome of your choices. Fittingly, it offers little in the way of answers to its biggest questions, with the consequences of its spectacular conclusion still somewhere beyond the receipt horizon.
I loved this book. I consider it an important contribution to the body of fiction depicting utopian Archipelago futures, as well as a keen warning about the challenges that an ontological earthquake like emulated minds might pose even to our best hopes for alignment.
It is of course possible that Bach's music is, in fact, not the voice of God, nor does it embody timeless and transcendent aesthetic truths, and altering a passage of Bach may not be an insult to the union of Truth and Beauty. But as a violinist, that framework of thought is written in Elara's bones. Her aesthetic understanding of Bach is as close to her spiritual/creative/aesthetic/epistemic core as anything can be. Think: lifelong devout Muslim suddenly discovering they suddenly keep making an instinctual change of rhythm or phrasing during their recitation of a passage of the Qur'an they've known since childhood. And some part of them feels the change is an improvement. That it's more true. This might provoke a foundational identity crisis.
When author Joel Borgen—in an act of eagle-eyed marketing marksmanship—contacted me as an "unusually plausible reader" for his debut novel, I grudgingly had to agree. Deploying a little tasteful and well-deserved flattery and then suggesting that
might be relevant to my interests, he had me dead to rights. I stepped into the baited trap eyes-open, expecting to have, at best, a mildly bad time.
Instead I discovered a book that instantly found a place on my bookshelf of favourite speculative SF novels.
Summary (no spoilers)
From Amazon:
So, yes, it's hard-SF, but the thing we're getting hardest about isn't the usual rocket engines or space wars or ML architectures themselves, so much as the semi-ambiguous web of institutional structures that now govern and control living humans and Ems (emulated humans) alike. It's a kind of limited-information thriller about acting under profound uncertainty as the opaque and mysterious moral and legal machinery of governing Utopia reveals itself slowly through the eyes of an entry-level participant who cannot stop testing boundaries and picking at stitches, waiting all the while to be crushed.
Her increasingly dangerous and consequential explorations trace the outlines of a paradoxical world in which ubiquitous surveillance and totalitarianism coexist with hard-coded constitutional values of consent and autonomy. Eventually this raises chilling questions about how our definition of the Good (and some of our best-case alignment outcomes) might survive contact with phenomena that challenge the underlying ontological assumptions on which that definition relies.
Detailed review: (SPOILERS)
'The Spike' was an intelligence explosion that led to a devastating AI war which humanity 'won' with the emergence of The Steward, an AI system that halted the destruction of the biosphere and now presides over a wildly utopian post-scarcity Archipelago-style multiculture of human-preference-satisfaction, allowing a thousand blossoms to bloom as diverse human communities pursue their own visions of the Good, built on a foundation of guaranteed peace, individual autonomy and consent.
The dream, right?
Elara
gets her owlis invited to attendSpace HogwartsThe Fulcrum Institute for Civic Leadership, a mysterious institution for training (what are essentially) academics and public servants/functionaries in the machinery of semi-participatory governance of human affairs.Elara's a Big Nerd who would read the Fulcrum school rulebook from cover to cover, except... she lacks clearance to read some of the pages... and she lacks the clearance to know which pages are above her clearance... and she lacks clearance to know who has what clearance... and her query has been logged... and her request to see that log entry has been logged... and would she like to avail herself of a voluntary INDIVIDUAL REFLECTIVE SESSION — 45 MIN via the Cognitive Safety division? Y/N? Her response will be logged. Not responding might result in a further "opportunity for structured self-reflection in a supportive environment" for her and those closest to her.
Perhaps she will "wash out", like some of her classmates.
The institution of Fulcrum seems actively concerned about her cognitive safety and intimately monitors her every reaction as she follows a curriculum that promises to expose her to "developmental hazards or adversarial memetics" while she navigates a series of "controlled epistemic shocks."
Despite all this surveillance and overbearing pastoral care, students do still enjoy considerable freedom, and can (they are assured) access meaningful privacy. Overwhelming surveillance and paternalism coexist with apparently real constitutional constraints, hard consent rules and independent human actors. Nobody can cleanly tell which constraints are genuine, which are governance theatre, which emerge from treaties, which are imposed by The Steward, and which are humans nervously administering a world they no longer understand. Like compartmentalised state secrets, much institutional knowledge is need-to-know only, and no single person or team gets to grasp the full shape of it. Except for the top brass, i.e. The Steward. Whatever that actually is...
Much of the thrill and growing sinister atmosphere of the book rests on the tension of just knowing in your gut that beneath the smiles and the care language, The Steward's system must rest somehow on coercion (in at least some cases), knowing that Elara is constitutionally incapable of staying in her lane, and waiting for trouble to erupt. But the killer blow never seems to come for Elara, in a way that becomes stranger, more suspenseful and demanding explanation as the book continues.
The cosmic horror of the book sets in as we gain fleeting glimpses into the events of the War and the compromises involved in winning it. Tiny fossilised echoes of the desperate choices made by humans in the final moments. Little glimpses of where the boundaries and barriers really lie in this new system, what 'the stick' of enforcement and coercion looks like when it occurs, what forms scarcity and adversity now take, how much has changed and how much has been lost.
Or might have been lost. Who really knows?
Academics and staff at Fulcrum do not, themselves, necessarily even have answers to the biggest questions a smart student might ask if they dared. Astronomers' main job now is to interpret the dying starlight to guess what The Steward might be doing out there. The Rediscovery Institute tries to piece together coherent accounts of the history of The Spike, while noticing that the details they're told vary depending on how the question is posed. People in senior positions all seem to have learned to use what affordances they possess to do good where they can, while embracing a kind of strategic epistemic helplessness. Restrained curiosity as survival mechanism.
Characters
The book is also a charming and tender coming-of-age story. While I don't primarily read SF for the characters or their stories, this book sure does have 'em. If you want a book that's 10% about having a crush on your college roommate and 90% uncovering the vast cosmic horror you're both living within, then this book will please you. The story succeeds in adding tangible emotional stakes to the abstract moral conundrums the characters are grappling with.
Borgen uses a fairly standard YA premise for the novel: five podmates from wildly different cultures must navigate their first year at Fulcrum. I personally am a jaded middle-aged man and I expected to encounter various embarrassing YA failure modes, but Borgen handles it all well. The interpersonal drama and ethical differences between the podmates turn abstract philosophical questions into actual conflicts between people who care about each other. The book threads a needle nicely: the characters are neither normies nor hyper-autists. The vibe is neither unrealistically sterile nor too lascivious. Sure, Kai does have nice abs and Serin
is a Veelalacks inhibitions in the sauna—but the author's gaze lingers only a tasteful moment. The second look is haram.The unfolding love story seems to raise the stakes as Elara blindly runs towards what we assume is a brick wall. By the time the podmates are forced to make genuinely consequential decisions together, their philosophical disagreements become entangled with friendship, loyalty, fear and love. And, usefully for the plot, they are still young enough, intense enough and convinced enough of their own reasoning to make some spectacularly reckless choices.
Consent and Continuity of Self (HEAVY SPOILERS)
Students are openly taught at Fulcrum that technology exists to create/delete/modify memories, personality traits, preferences and experiences. They encounter demonstrations of these capabilities while having their reactions closely monitored.
Of course, nothing can be done to a person without their informed consent, so these mind-modifying techniques become a useful toolkit for people wishing to improve their own lives, recover from grief and mental illness, and experience new pleasures alone and with others. Consent can never be violated, except in the most extreme cases of exigency...
Problem is, that doesn't help much if you're a spur instance of a template Em, whose template has consented on your behalf to your brief existence and eventual deletion.
Your deletion anxiety has been logged and will be reflected in the session record.
Crucially, even for a biological human, consent gets strange once preferences themselves can be modified. After an edit, the resulting person may sincerely endorse the change, but that endorsement is no longer independent evidence that the change was legitimate: the edit may have removed precisely the preference that would have wanted it reversed. Worse, if memory and motivation can also be altered, later-you might have no way to know whether this has already happened. I don't recall if the book broaches this, but there are also problems where, ship of Theseus style, a person might consent to many sequential personality edits, each one leaving them more inclined to consent to the next. Thus, the second-to-last iteration consents to something the original might never have wanted.
She realises that humanity as a whole might face a ceiling like this.
Much of the heart of the novel revolves around Elara and her podmates learning the extent of the 'Identity Integrity Concerns' that might arise as a consequence of these technologies. It lands differently for each student, and they must resolve their disagreements without making their concerns too legible to those watching.
This may be a post-scarcity society, but the amount of compute allocated to humans is still finite, and so political economy remains a question of "who gets what, when, how." The structure of governance under The Steward is "an apparatus for deciding which minds stay running."
Ultimately, they must together decide what it really means for humans to survive. Is a scenario where humans continue living but do not meaningfully impact the future light-cone really a form of soft-extinction? How much can abundant food and beautiful architecture compensate for an Em having its continuity of experience rationed because human-derived minds remain a rounding error beside The Steward's optimisation capacity? And what risk is justifiable in trying to take back a slice of the future, when humanity has already, somehow, preserved so much that is worth protecting?
Wireheading yourself with violin sonatas and selling your soul to the devil
Elara is a violinist, though she struggles to practise once she leaves home, and the instrument mostly sits neglected in its case in her room. During a computer-assisted recreational 'coupling session' with Kai using a 'crown unit' placed on her head, Elara reconciles with her violin and essentially wireheads herself with it. Physical technical limitations recede dramatically, and her interpretation of Enescu's Impressions d’enfance seems to flow directly from her intention.
She still experienced the creative impulses as her own, and did not experience the "foreign intention" she'd been warned of in the waiver when she signed into the session.
Later in the book (and after more violin-related cognitive restructuring that I will not spoil here), while playing Bach, she experiences a sudden and novel musical impulse. Her fingers spontaneously rebalance/reinterpret an inner voice in a line she'd played a thousand times before.
Then she has something akin to a heretical thought:
Then:
She no longer trusts the development of her own musical judgement. The choice of Bach here makes this especially profound: many violinists treat Bach with utter reverence—the music is taken as somehow axiomatic, self-sufficient, foundational. Not merely beautiful, but somehow true. Inviolable. Incontrovertible. The voice of God[1].
So something has changed in her very deepest perceptions of beauty and truth. That's not a disaster necessarily, altering your aesthetic judgement is sometimes called learning. But the horror is that she can no longer cleanly distinguish between learning and modification.
This question confronts all authors, musicians, or creatives of any kind. AI-enhanced creativity feels good, it unsticks our creative interpretations from our technical limitations, and it potentially unlocks deeper and surprising interpretations we might not have reached unaided. But once we've let it in, how much can we trust our own discernment?
Conclusion
This is largely a book about acting under profound uncertainty, and perhaps never knowing the outcome of your choices. Fittingly, it offers little in the way of answers to its biggest questions, with the consequences of its spectacular conclusion still somewhere beyond the receipt horizon.
I loved this book. I consider it an important contribution to the body of fiction depicting utopian Archipelago futures, as well as a keen warning about the challenges that an ontological earthquake like emulated minds might pose even to our best hopes for alignment.
It is of course possible that Bach's music is, in fact, not the voice of God, nor does it embody timeless and transcendent aesthetic truths, and altering a passage of Bach may not be an insult to the union of Truth and Beauty. But as a violinist, that framework of thought is written in Elara's bones. Her aesthetic understanding of Bach is as close to her spiritual/creative/aesthetic/epistemic core as anything can be. Think: lifelong devout Muslim suddenly discovering they suddenly keep making an instinctual change of rhythm or phrasing during their recitation of a passage of the Qur'an they've known since childhood. And some part of them feels the change is an improvement. That it's more true. This might provoke a foundational identity crisis.