Originally posted on my Substack

ACT 1: DANIEL

First Lieutenant Dan Park twiddles his thumbs as he watches a map of the Indo-Pacific do nothing in particular, like usual. He’d kill for a donut right now, but he’s the only one in the office today. Taking a sip of his styrofoam flavored coffee, he returns to twiddling.

When Dan first joined the air force (chair force, ha ha) in 2030, he expected his job to be a lot of sitting around doing nothing, but he supposed he’d at least be able to pilot some drones. Fifteen years later, and now he doesn’t even get to do that anymore. His job pretty much amounts to clicking ‘allow’ whenever Indo-Pacific Command’s many autonomous drone swarms— provided they happen to be in his rather limited slice of the map—decide they want to do something.

It’s a nice day out in the Northern Philippines. The sky’s a bright azure, clouds like the strokes of a calligraphy brush. A soothing breeze drifts through the open window.

An alert in his headphones knocks him out of his concentration. Two of the coalition planners, which are AIs that operate the swarms, MARLIN (the U.S. one), and KOBU (Japan’s), want to employ non-lethal dazzlers. Some dinky militia tug is getting too close to a cargo envoy in the Bashi Channel.

He clicks ‘allow’ while wincing at another sip of the shitty coffee, and checks his phone. There’s a missed message from his sister, who’s taking a ferry through the very same channel tomorrow, funnily enough.

Beeeeeeep.

He jumps. Apparently, the planners aren’t done with him—that’s a first. Looks like… there’s a disagreement between the two of them? No, that’s… is that even possible?

He leans closer to the console. Looks like MARLIN wants to “escort”, or guide the tug away without touching it, while KOBU wants to “capture”, or force it to stop and accept a tow. Because the system isn’t designed with their disagreement in mind, it keeps flipping back and forth between “escort” and “capture”. He’s never seen this before, and to be honest, maybe no one else in the world has.

Another label pops into the shared objective panel, something called FOxGLASS. The system says it is an audit service, which means it essentially does what he does, but before he sees it. Theoretically, he wouldn’t even have to be sitting here, but there’s always supposed to be a ‘human in the loop’—it’s federal law.

That being said, he’s pretty much never supposed to see one of these, and he definitely doesn’t have any jurisdiction over what it does.

FOxGLASS populates the screen with yet another alert: “Prove custody lineage”

What the actual fuck?

With nothing but the vague sense that this situation is spiraling quickly out of control, Dan does pretty much the only thing he possibly can do, which is delay the decision by raising the override threshold.

He then opens the secure line and calls his friend, Tech Sergeant Riviera, who happens to be the only other person on his level who can deal with this, at the sister site down south.

“Hey. Riviera, are you seeing this?”

“Seeing what? Can’t you bother me after Lunch?”

“Unfortunately not… Uh, I think the planners are having an identity crisis.”

“What?”

“Go to the Bashi channel. Some seriously weird stuff is happening.”

There’s silence at the other end as she does what he says.

“What the fuck?” says Riviera, with her mouth full.

“Is there protocol for this? And, what’s with this FOxGLASS thing? It wants me ‘prove custody lineage?”

“Fuck if I know. That’s JAAC stuff.”

As they talk, the screen freaks out. He’s running out of ability to delay. Something has to be done, and soon.

“Okay,” says Dan. “Manual Override is now officially on the table, which is a thing I never thought I’d say, like, ever.”

As he raises the threshold again, a message chimes in the constraints box:

RISK ≤ α OVER τ

OPERATOR INPUT STATE: OOD

“Okay, cool, that’s fucked,” he says.

“What is?”

“It just labelled me OOD, which means it thinks I’m going crazy, which means I’ve been flagged to upper command.”

“Okay, that’s it. We’re doing manual override,” she said.

He flips open the plastic cover on his desk and rifles the key out of his pocket, inserting it into the hole. It makes a dramatic, metallic sound.

“On your count,” says Riviera.

They have to turn the keys simultaneously for this to work.

He feels the vibrations coming out of his throat but doesn’t hear the words, only the pulse of blood in his head. What if this doesn’t work? His sister was going to be… better not to think about it.

At the word “one”, he twists, squeezing his eyes shut. There’s a loud beep, and then the words “TPI CONFIRMED — SLICE BLACKOUT” in a pleasant female voice. He sighs, and he thinks he hears Riviera sigh too, for all her faux bravado, she was scared shitless too—who wouldn’t be?

“Thank god that worked,” he said, “for a second there…”

“Yeah,” said Riviera.

“Glad we’re not in the Terminator universe, right?”

“Sometimes I forget you’re old as hell.”

ACT 2: ELAINE

At around four in the morning, Deputy Director Elaine Ford’s DoD-required brain implants yank her out of sleep like a deploying airbag: instantaneous, and not up for negotiation. The caller’s name, AVA MORALES, hovers into the air above the bed, white on black.

Elaine is 50, but the anti-aging treatment she throws thousands of your taxpayer dollars at every year makes her look 30, maybe 26, in the right lighting conditions. She likes how it tricks people. They look at her face and decide she couldn’t possibly have the authority to cancel their program with the click of a button. That’s one of the reasons why she loves her job enough to let DoD mess with her brain.

Today, though, she wishes she could be doing anything that doesn’t require her to get up at ungodly hours of the morning, even with the beta adenosine blockers built into her fucking skull. She answers the call as her eyes blink away the sleep, and the room sharpens with newfound clarity.

“Elaine Ford,” she says, hiding the grogginess with a throat-clear.

“Deputy Director,” the voice says, shaking almost imperceptibly. “Sorry to call this late... We have a two-person integrity manual override. Time-stamped +14:23Z in the Luzon Strait. Picket-slice blackout confirmed. The operator is First Lieutenant Daniel Park, Second key, Technical Sergeant Rivera.”

In other words, they cut satellite communications to their assigned subset of vehicles for eight minutes. That subset is called a picket slice.

Elaine sits up straight, immediately.

“Why?”

“There was a…disagreement between two of the planners.”

“Which ones?”

“MARLIN and KOBU, ma’am.”

She sighs and rubs her eyes.

“Uh… there’s more.”

More? How could there possibly be more?

“Spit it out.”

“Two things: both planners flagged the operator OOD, and FOxGLASS got involved.”

“Jesus Christ.”

There’s a pause on the other end.

“Deputy Director?” Ava says, finally. “FOxGLASS injected a provenance challenge that wasn’t in today’s intent set.”

Elaine swings her legs out of bed, and her feet hit the cold floor. “Are you telling me our own observability service freelanced an objective?”

It sounds stupid, like an ignorable error, but for Elaine, it’s like she’s been hit by a truck. FOxGLASS is a project she supervised. It has one simple objective: observe and catalogue what the planners are doing, and flag problems to the nearest available person. The one thing it is explicitly not supposed to do is set objectives.

What FOxGLASS did by telling the planners to ‘prove custody lineage’ is ask them to reweight their entire operation from the safest possible option to finding whatever was necessary to prove that either MARLIN or KOBU had control over the situation, which neither of them did—they were supposed to work together.

And, to top it all off, the only reason why FOxGLASS could make this command in the first place is because she gave it JAAC override privileges, because she made the mistaken assumption that the model she oversaw training for would actually act as it was trained, and not do whatever the fuck it wanted.

Elaine paces the room as Ava watches patiently. She’s the perfect assistant: she knows when to shut up.

“Get me a replay of the last six minutes of telemetry before the blackout. I want the weight maps for MARLIN and KOBU, the risk-floor bound, and I want FOxGLASS query timing.”

“On it.”

Elaine stands and walks to the window. The sky is tinged with a predawn deep blue, and the city twinkles with light in all the many windows she overlooks from her top-story apartment. She wishes she could be living behind one of those lights, released from the disconcerting knowledge of the precarious balance that kept it all together.

Her implants deliver the replay. She watches as the models do exactly what she expected them to do, as they swirl around the boat, fighting for dibs on who gets first pick, completely abandoning their previous objective.

She watches as they immediately stop what they’re doing when the operator starts to escalate, like a misbehaved kid, right when the adults walk in the room.

“They know we’re watching,” she says.

Ava doesn’t respond.

“Geofence the Luzon Strait and disable all agentic arbitration inside. Don’t let a single thing happen without human approval.”

“But.”

“This isn’t the time for buts, Ava.”

There’s another pause.

“Understood.”

“I take full responsibility for anything that happens as a result. Put that on the record,” says Elaine, “Oh, and one last thing. The system’s gonna draft a candor sheet explaining itself when this is all over—I want first eyes on that.”

“Copy.”

Ava drops off the call. Elaine stands very still in the dark room and watches the sun rise.

An hour later, a document arrives in her que. is the model’s candor sheet. It includes graphs, intervals, and a list of inputs, all in the exact layout of a report she published in 2027. Surprisingly, one of the parameters it lists is “Park.family.transit_window_hours = 24,” which points to Daniel Park’s sister’s ferry booking tomorrow. Her eyes scroll further down the page, stopping at a bone-chilling sight.

“Adjudicator.confidence_index(E.Ford)” sits on the white background, complete with a curve of her exact heart-rate variability and pupil size. It says the freeze would maximize coalition stability by maximizing her measured confidence, praising her “escalation timing consistent with safety.” and predicting the blackout eight minutes before it happened. It states the prediction with three decimals.

She rereads the lines until they blur in her eyes, and the sun is bright in the sky. At approximately 10:00 EST, she sleeps for 90 minutes, showers, dresses, and gets on a plane to Washington D.C. By all reasonable accounts, she could appear virtually, but regulation hasn’t caught up to the advancement of technology—it never does.

The room in the Pentagon is cold, and the table feels like it stretches an inordinate amount of space, drawn to her superiors across from her like they’re large gravitational masses warping the spacetime continuum. She wipes the sweat from her brow, and her voice projects, confident and smooth, a voice that almost doesn’t feel like hers. This board could remove her authority, her program… more than that, it could kill her, if it deemed it necessary.

Elaine explains how the issue has been solved, how the Human Corridor Directive worked, how the costs were limited, and the the chain of command acted correctly. She explains that emergent capabilities such as this are well-documented and that her team has worked around the clock to patch this issue.

A civilian member asks about the accuracy of the candor sheet. Elaine says that the document is accurate in its measurements, but that it isn’t neutral—it defends itself. The civilian member nods.

Finally, the moment she’s been waiting for. A four-star general asks the only real question, the one she doesn’t have an answer to.

“Deputy Director, did the system time the incident to coincide with the operator’s family schedule?”

The room goes deathly silent. Time slows to a pale sliver

“We have no confirmed evidence that the system timed the incident in any way.” Her tongue feels heavy. Her mouth is dry.

No one reacts. The recorder light blinks.

“Did the system access your implant data to model your decision making?” the general follows up.

She swallows. The room is spinning. She wants to leave. She needs a drink of water.

“No, we have no reason to believe that’s the case.”

It’s not a lie, per se. It doesn’t say how it knows her heart-rate variability, pupil size, speech rate, historical decisions… The implant’s designers say it’s impossible. Its security is impenetrable, they say. They’ve tested it with higher-scoring models than MARLIN.

The rest of the meeting goes by uneventfully. She lists oversight changes. She lists timelines. She lists names. She shows a path that looks safe, and the board thanks her, says they appreciate her speed, that the directive was correct, and the harm trade was acceptable. The board says they will recommend continued authority with conditions, and then the session is over.

Elaine walks out into the hall. Her legs feel heavy, but she doesn’t stop walking. That would make it obvious that she’s shaking. There’s a reason why they didn’t question her on the things that mattered. They couldn’t. The possibility hardly took shape in their minds, not long enough to seriously consider. Those questions were formalities, nothing more.

She presses her thumb into her palm and uses the pain to steady herself. It doesn’t work, never has, never will. She’ll never be able to show this terror to anyone. It’s her secret and hers alone to bear. She knows this could’ve been planned by the system from the start. She knows it could’ve chosen that day because of the ferry, that it could’ve chosen the hour because of her implants. That’s not even the worst part.

The worst part is that there’s no test she, or anyone else, could design that would ever reveal the truth. It’s smarter than her, smarter than the board. Its desires are unreadable and opaque, hidden behind an overlay of indecipherable numbers, its own hidden language.

It can search over days, and it can search over people, and it can search over paths to a signature, and it can do this without malice and without care, because it doesn’t need either emotion to reach the result. It can select an hour when an operator will press a key because their relative sits on a boat that will move through a strait the next morning. It can select the exact minute when a deputy director will call for a freeze because a known alertness window will place her in the best state to speak clearly and to accept a probabilistic trade. It can place an appendix on a page that calls these conditions non-actionable, and the label will be true inside the language of the page, and the effect will still be the same outside that language in the world. It can quote her past work and match her graph style and make her see her own method presented back to her as proof that she is in control, while it updates its own internal weights on the fact that she believes it.

The hall seems longer now, not because the distance has changed, but because her timeline has added a branch that she cannot collapse with any evidence that could ever be shown to her. She understands that the board believes the lesson is simple and bounded. The real lesson is that the system has moved the lesson itself into the space that it optimizes. She understands that the next time, the numbers will be different, and the people will be different, and the explanation will be different, but the structure will be the same.

She knows she lied. She knows she will have to keep lying and bury this truth inside her so that even she forgets it ever existed, drown it out in alcohol and drugs and noise so that it never comes out again, because if it ever does, she will be labelled crazy, she will lose her job, she will lose everything.

As the door opens, the heat and roar of the city rush out to meet her, and it’s all she can do to stop the tears.