Hardware hacks can be "demonically" clever (or just normal amounts of clever) and it is somewhat distressing to not see people take it more seriously even in the deep fictional future.
I'm coming around lately to the idea that the way to go is artisanal hardware made by friendly monks or something? Chip designs printed on 8.5x11 transparencies. Megahertz speeds. For some signal processing kinds of things the security seems like the most important factor.
It feels like it might work like covid testing/vaccines in practice: by the time "the system" does it, it will have been too late, for too long, to avoid enormous pain?
In the meantime, the writing is dense with ideas and I seek it out on purpose. Thank you for authoring and for publishing here!
two her brain
This gave me the chills. I have never thought about digital brain modification safety before, although now the idea seems obvious. Wonder what else am I missing.
I have fixed the typo; "two her brain" is now "to her brain". There's all sorts of stuff that can go wrong when you're modifying human brains.
"Two" was a typo. I have fixed it.
In case anyone missed what was going on underneath the technical gobbledygook, here is a clarification:
The connection was supposed to be bidirectional. That way Trinity's personality would overwrite the collective's personality in equal proportion to the collective overwriting hers.
She signed up to become one sixth of a being with six times her consciousness. Instead, the collective modified the interface to create a one-directional connection. They overwrote Trinity's personality without letting her overwrite its own at all. It stole her body and brain.
Makes me wonder whether she was the first victim, or this is how the collective operated since the very beginning. The first member overwrote the second, then together they overwrote the third...
Just making sure that I understand it correctly... Trinity still retained 1/6 of her original personality, right? That is, if we assumed that the interface was modified already by the first member of the collective, then the first member would remain 100% their original self, the second one would be 1/2 A + 1/2 B... so the two-member collective would together be 3/4 A + 1/4 B... the third member would become 2/3 × 3/4 A + 2/3 × 1/4 B + 1/3 C = 1/2 A + 1/6 B + 1/3 C... now with Trinity the entire collective is cca 3.5 A + 1.2 B + 0.6 C + 0.4 D + 0.2 E + 0.2 F. Assuming that each new member gets (N-1)/N of their personality overwritten by 1/N of each current member, the collective would have to grow to 55 members to make Trinity regain the full size of her personality. (Which is not going to happen for in-universe reasons: "can sync only a handful of brains".)
As I understood, her keeping 1/6 of her personality was not a technological limitation but a part of the contract. Since they broke the contract, I assume they erased however much they wanted, possibly all.
Also, the 'they' is another assumption that comes from their discussion only. For all we know, 'they' is a single mind puppeteering all the other bodies.
And since this series is about AGIs gone rogue, I assumed it likely that the puppet master was a rogue AI, or a human agent/associate of one.
As I understood, her keeping 1/6 of her personality was not a technological limitation but a part of the contract. Since they broke the contract, I assume they erased however much they wanted, possibly all.
Yes. This is exactly correct. If we are to split hairs, Trinity keeping 1/6 of her personality was an unwritten fact the contract took for granted to be true. Trinity keeping 1/6 of her personality used to be a technological limitation. At the time this story takes place, it is no longer a hard technical limitation. The state's regulatory system has failed to keep pace with technological advance.
Also, the 'they' is another assumption that comes from their discussion only. For all we know, 'they' is a single mind puppeteering all the other bodies.
Yes. Muddying the waters even further, the use of the word "they" is a culturally-dependent term with political implications. The meaning of pronouns changes when you start hooking people together into hiveminds.
And since this series is about AGIs gone rogue, I assumed it likely that the puppet master was a rogue AI, or a human agent/associate of one.
No comment. 🙂🤖
That's a good question. Your division equation of her retaining 1/6 of her original personality is a perfectly reasonable interpretation. I might even retcon it in. I hadn't thought about the numbers that precisely. My original intention was that Trinity's whole personality was erased in the parts that got hooked up to the collective but that hardware limitations keep less connected parts of her brain relatively unaltered. Like, the part of her brain which keeps her heart beating needn't be overwritten.
It also might make sense for the collective to keep some of her skills and memories around, especially valuable areas of expertise the collective otherwise lacks. How the assimilation process works may change over the course of the story as technology advances.
I assumed a simple hack (the interface remains as it is, only the cable going in the opposite direction is cut/blocked) rather than some more complex reingeneering of the interface. But this is an unwarranted assumption on multiple levels -- the way the interface works (a simple cut of cable will block copying in one direction without raising an alarm), the technical skills of the collective (if they are qualified enough to do a simple hack and avoid detection, it is not too unlikely to assume they could do a complex hack, too).
My interpretation on first reading was the same as your original intention. I agree with the second paragraph, that seems reasonable.
(I'm writing this vaguely because I don't remember syntax for spoilers, and can't be bothered to look it up)
Typo watch:
Collectivization is so safe that days people with epilepsy often cure themselves by joining a collective.
should be
Collectivization is so safe that these days people with epilepsy often cure themselves by joining a collective.
"two" should be "to".
Seeing references to the Matrix in terms of the hard-wiring of computing technology directly into human neurology, makes me think going all in might be good for verisimilitude.
I think the amount of glucose and neurotransmitters needed to accomplish the rewiring of a 19 year olds brain would require something like a bath of the sorts Neo woke up in when we first saw him in the real world. Or maybe something with a more compact form factor, like the helmet the main character in the Abyss used to breathe liquid for going super deep in the ocean. That scene when he first started breathing in the liquid was tense to say the least.
Issues like the amount of heat generated through the constant excitation of and creation of new neuronal connections might require active cooling, and the process of pruning old connections in the brain might require literal brain washing.
Unless this scene is happening in a simulation? If it was a simulation, that would give you some extra room to explore how some AI rely on simulations to model their 'understanding' of the world outside their networks and could lead to some interesting scenes.
Three humans sat around a table. The notary was unaugmented. He wore a charcoal business suit. The ancillary wore a silk paisley print top zippered at the back. A thick cable extended from the back of her head to the collective's mainframe. Trinity nervously fondled the virgin socket embedded in the occipital bone of her own skull.
"Please confirm your DNA," said the notary.
Trinity pricked her thumb on the bloodometer.
"You are Trinity Sariah Rees," said the notary.
"You don't say," Trinity rolled her eyes.
"Nineteen years old. Mormon…" said the notary.
"Ex-Mormon," corrected Trinity.
"Please speak into the microphone. We need an unambiguous record of your consent," said the notary.
"This is ridiculous. It's my body and mind. I should have the freedom to do what I want with it," said Trinity.
"Informed consent is important," said the ancillary, "We don't want to get into legal trouble. Tomorrow you will feel the same way."
"Which is why we need consent now. It won't count after the procedure," said the notary.
"I already had the surgery. All we're doing is plugging me in," Trinity crossed her arms and legs.
"The government is prejudiced against transhumans," said the ancillary, "It could be worse. We're lucky all they demand is paperwork."
"Let's get this done quickly," said Trinity.
"Are you really ready to have five sixths of your personality erased?" said an ancillary.
"That five sixths will be equally divided among you guys. Are you ready to have one sixth of your personality become me?" said Trinity.
"It sounds like you are familiar with predictive processing and connectome-specific harmonic waves," said the notary.
"I have wanted to join a collective since I was thirteen. I keep up-to-date on the newest research," said Trinity.
"Then I don't need to explain how modern neuroscience is based on the idea that synapse weights modify the topology of your brain like how gravity bends spacetime," said the notary.
"Do you always use general relativity metaphors when explaining things to people?" said Trinity.
"I didn't want to insult your intelligence," said the notary.
"It's about time someone recognized it," said Trinity.
"We do too. We can sync only a handful of brains before the global workspace fragments. We must be selective about who we assimilate. High -factor is a priority," said the ancillary.
"Anyway," said the notary, "The primary purpose of the human brain is to create a predictive model of its environment. Everything else is secondary."
"Technically it is the primary purpose of consciousness to create a predictive model of its environment," said Trinity, "The human brain does other things too. What really matters is the connectome. When you link multiple connectomes together cybernetically you combine their individual topologies into a single larger topology. It is the opposite of a corpus callosotomy. Software in the synchronizing mainframe prevents epileptic seizures. Collectivization is so safe these days people with epilepsy often cure themselves by joining a collective. Collectivization is simple technology. If it weren't for runaway resonance you could do the whole thing with analog electronics. "
"That's the most technical summary I've heard from any client so far. It sounds like you know exactly what you are doing. I hereby confirm you have informed consent. Just sign this document and you are certified to assimilate," said the notary.
Trinity signed.
"Congratulations," said the notary, "You are legally part of the collective. Everything else is just software."
Trinity sat in the collective's assimilation chair. All five ancillaries of the collective stood around her. Their tether cables were decorated in paints and gemstones. They secured her wrists and ankles with padded shackles. A clamp stabilized her head. A belt went around her waist.
"We don't want you to hurt yourself if there are muscle spasms," said an ancillary.
"Just give me the mouth guard," said Trinity.
An ancillary inserted it.
"Ready?" said an ancillary.
Trinity was too safely secured to nod her head properly. She managed an upward twitch.
The collective began its sync. Trinity received five universes of semantic data from the other ancillaries. Her brain emulated them. The collective personality overwrote her own.
Trinity tried to sense her own personality overwriting theirs. Communication was a one-way street. The collective had hacked the interface to transmit data to her brain but not from her brain. She struggled against the shackles. She tried to scream but her mouth was clamped shut.
The collective released her head after six hours. It compelled her to eat and drink. The assimilation resumed.