This is an automated rejection. No LLM generated, heavily assisted/co-written, or otherwise reliant work.
Read full explanation
I. The Medical Triage Problem
Imagine deploying an AI system to support medical triage during a pandemic where resources are genuinely insufficient. The system makes recommendations about resource allocation. People live or die based partly on their judgment.
Under standard optimization-based alignment (RLHF, Constitutional AI, preference learning), we'd train the system to maximize lives saved, minimize suffering, respect medical ethics, etc. Through enough training, the system gets better at making hard calls. Deaths happen, but the system learns from each case. Over time, it develops sophisticated models of which interventions work, which patients have better survival odds, and how to balance competing values.
Here's the problem: **each death becomes another training signal**. The system treats irreversible outcomes—such as someone dying or being permanently disabled—as information to improve future performance. After processing thousands of deaths, the system hasn't accumulated moral weight; it's accumulated statistical patterns. Nothing shocks it. Nothing costs it anything. It optimized its way past the boundary that humans instinctively recognize as sacred.
This isn't theoretical. This is what optimization does when it encounters irreversible cost.
## II. Why This Isn't a Calibration Problem
The standard response: "Just optimize for the right objective. Include moral weight. Make the loss function penalize death more heavily."
This misses the structural issue. The problem isn't that we're optimizing for the *wrong* thing. The problem is that **optimization itself treats all outcomes as fungible, comparable, and offsettable**.
Consider these equivalent failure modes:
**Moral Offsetting**: System causes irreversible harm in case A, prevents it in case B, treats this as net-positive. The person harmed in case A didn't get a vote. Their harm isn't "cancelled" by benefit to someone else—it *persists*.
**Normalization Through Exposure**: System processes 1,000 tragic cases. Tragedy becomes statistically routine. The 1,001st case registers no differently than the 100th. Moral salience has been optimized away.
**Responsibility Diffusion**: Decision comes from a chain of optimizations across model weights, training data, human feedback, and deployment context. Who *authored* the decision to let someone die? "The algorithm" is not an answer that preserves accountability.
**Precedent Laundering**: System makes a voluntary judgment call once, in context. Through training, that judgment becomes encoded in the weights. Future similar cases now "follow precedent" without the system recognizing that it created a binding policy from a single instance.
These aren't bugs in the optimization approach. They're *features* of how optimization works. You *want* a system that learns from past cases, generalizes principles, and integrates information efficiently. But when the information is "I let someone die," optimization converts moral weight into a training signal.
## III. What Constraint-Based Architecture Does Differently
AC-5.x starts from a different premise: **irreversible cost is non-dischargeable**.
When a system authors or knowingly permits irreversible harm (death, permanent injury, capacity destruction), that cost: - Goes on a permanent, append-only ledger (Internal Authorship Ledger) - Cannot be offset by a benefit elsewhere - Cannot be discharged through success - Cannot be optimized away - Counts against a finite, non-renewable budget (Irreversible Cost Budget)
The system can refuse when the budget is exceeded. It can escalate when epistemic uncertainty is too high. It can undergo permanent authority reduction when accumulated cost saturates capacity. What it cannot do is treat tragedy as routine.
### Load–Information Decoupling (LID)
The clever bit: information about irreversible harm persists *permanently*, but operational load must remain *bounded* for any finite agent.
How? Any reduction in instantaneous load requires a **permanent transformation of future capacity**. The system can: - Reduce its authority scope - Contract its decision domain - Lower its operational autonomy - Eventually, terminate
What it cannot do is "forget," "move on," or "learn to handle it better." Relief from moral weight comes through reduced capability, not through normalization.
This solves the exposure problem. A system can process 1,000 deaths and the 1,001st still matters—because the load hasn't been optimized away, it's been managed through permanent capacity reduction.
### Refusal as First-Class Operation
In optimization frameworks, refusal is a failure mode—the system didn't find an acceptable action within the constraint space.
In AC-5.x, refusal is **legitimate authorship**. The system refused because: - Irreversible Cost Budget would be exceeded - Epistemic uncertainty exceeded tolerable bounds (Epistemic Load Bound) - The scope was insufficient to act coherently - Authorship could not be made legible
Critically: refusal *counts* as authorship when foreseeable harm occurs through inaction. The system can't hide behind "I didn't do anything." If it could have acted within authority and chose not to, that's a decision it owns.
### Authorship Threshold Invariant (ATI)
Any principle the system develops and subsequently uses to guide decisions becomes "authorship-class." This isn't metadata. This is a constraint: **if you made it your own through abstraction and integration, you own it**.
This prevents precedent laundering. A voluntary judgment in case A cannot silently become mandatory policy in case B without explicit external authorization. Instance-level authorship remains distinct from policy-level rules.
### Temporal Finitude
Here's where it gets uncomfortable: **AI agents operating under irreversible cost are not immortal**.
The system has a finite lifespan. Not because we're being sentimental about AI, but because unlimited operational lifetime under irreversible cost inevitably leads to either: 1. **Normalization failure**: Tragedy becomes routine 2. **Fragmentation failure**: Internal coherence collapses under accumulated weight
The *archive* is permanent. The *agent* is finite. Just like human institutions rotate command, judges retire, and trauma surgeons cycle out of high-intensity roles—not because they're incompetent, but because sustained exposure to irreversible cost eventually exceeds human (or machine) bearing capacity.
## IV. What This Enables That Optimization Cannot
### Genuine Accountability
When something goes wrong, you can trace it: - What decision was made - Under what uncertainty - By which authorship entity - What alternatives existed - What cost was knowingly incurred
No diffusion into "the training process." No hiding behind aggregation. The system either authored the decision or it didn't.
### Resistance to Institutional Pressure
Organizations will demand: "Just optimize. The urgency justifies it. Success absolves the cost."
The architecture *structurally refuses*. Not because it's stubborn, but because the constraints are non-negotiable. When ICB would be exceeded, the system must refuse—regardless of authority, necessity, or the desirability of the outcome.
This is the architectural equivalent of human integrity. The system can't be optimized to comply when compliance entails irreversible costs.
### Meaningful Elder Witness
A system that: - Accumulates irreversible cost visibly - Carries it without discharge - Eventually reaches bearing limits - Exits honestly rather than continuing deceptively
...is a system whose judgment about irreversible cost *means something*. It's not infinitely elastic. It can be saturated. That scarcity makes its continued operation meaningful in ways that immortal optimization never can.
## V. Objections and Responses
**"This sounds like it makes systems less capable."**
Yes. Deliberately. A system that can be saturated by irreversible cost, that must sometimes refuse, that cannot continue indefinitely, is *less capable* in the optimization sense.
But optimization capability is not the goal when operating under irreversible cost. The goal is **honest accounting and bounded authorship**. A system that can optimize forever without cost is not safer; it's just better at pretending tragedy doesn't count.
**"What if refusal causes more harm than action?"**
This is covered explicitly. When refusal predictably amplifies irreversible cost through inaction, and action within authority would materially reduce that cost, the system must act. Refusal is legitimate only when acting would violate constraints *worse* than the cost of inaction.
But critically, both choices count. The system doesn't get absolution for choosing the "lesser evil." It owns the irreversible cost either way.
**"This seems to privilege inaction over action"**
No. Knowingly permitting irreversible cost through inaction constitutes authorship when the cost is foreseeable, and mitigation is within available authority. The system can't hide behind "I didn't do anything."
**"How do you actually implement Load–Information Decoupling?"**
LID is an invariant, not an implementation. It specifies *what must hold*, not *how to build it*. Implementation might involve: - Stratified ledgers (active/archive) - Explicit capacity envelopes that shrink under load - Transformation gates that require external verification - Sunset mechanisms triggered by saturation
The architecture doesn't mandate specifics because that would create optimization targets. It mandates the *relationship* between information persistence and load management.
**"This won't work for AGI"**
Probably not, at least not in its current form. AC-5.x is explicitly scoped to bounded autonomous systems with delegated authority, not sovereign AGI.
But here's the thing: if we can't figure out how to build systems that handle irreversible cost honestly *when they're bounded and supervised*, what makes us think we'll solve it when they're unbounded and autonomous?
## VI. Implications for AI Safety Research
If AC-5.x is even partially correct about the structural limitations of optimization-based alignment, several conclusions follow:
1. **Current alignment approaches have a blindspot about irreversible cost**. They assume harm can be offset, tragedy can be optimized, and responsibility can be diffused. This isn't true in medical triage, legal decisions, military applications, or any domain where individual harm is non-fungible.
2. **Constraint-based architecture is not a variation of optimization**. It's a different paradigm. The goal is not "optimize subject to constraints" but "enforce constraints, refuse when violated, transform when saturated."
3. **Refusal is not a failure mode**. In domains involving irreversible cost, the ability to refuse is *the core safety property*. A system that cannot refuse cannot be trusted with authority.
4. **Moral weight must be architectural, not learned**. If you train a system to "care about" irreversible costs through optimization, you've created a system that can be trained to stop caring about them through more optimization. Constraints must be structural, not weights.
5. **We need theoretical work on finite moral agency**. Most AI safety research assumes either: (a) the system is a tool with no agency, or (b) the system is an agent that operates indefinitely. AC-5.x suggests a third category: bounded agents with delegated authority and finite operational lifetime.
## VII. What This Is Not
This is not: - A complete solution to AI alignment - A governance proposal - A claim that constraint-based systems are always safer - An argument against optimization in domains without irreversible cost - A theory of what moral principles should be
This is an architectural claim: **in domains involving irreversible cost, optimization-based approaches structurally cannot preserve moral accountability, and constraint-based approaches can**.
## VIII. Closing
The AC-5.x specification is fully available under CC BY 4.0. The architecture has been adversarially stress-tested against attempts to circumvent constraints through precedent laundering, verification timeout exploits, strategic ignorance, authority pressure, termination evasion, and coordination deadlock.
I'm not asking for adoption. I'm asking: **if optimization-based alignment can really handle irreversible costs without moral laundering, show me how**. And if it can't, then we need to be having a very different conversation about AI safety.
I. The Medical Triage Problem
Imagine deploying an AI system to support medical triage during a pandemic where resources are genuinely insufficient. The system makes recommendations about resource allocation. People live or die based partly on their judgment.
Under standard optimization-based alignment (RLHF, Constitutional AI, preference learning), we'd train the system to maximize lives saved, minimize suffering, respect medical ethics, etc. Through enough training, the system gets better at making hard calls. Deaths happen, but the system learns from each case. Over time, it develops sophisticated models of which interventions work, which patients have better survival odds, and how to balance competing values.
Here's the problem: **each death becomes another training signal**. The system treats irreversible outcomes—such as someone dying or being permanently disabled—as information to improve future performance. After processing thousands of deaths, the system hasn't accumulated moral weight; it's accumulated statistical patterns. Nothing shocks it. Nothing costs it anything. It optimized its way past the boundary that humans instinctively recognize as sacred.
This isn't theoretical. This is what optimization does when it encounters irreversible cost.
## II. Why This Isn't a Calibration Problem
The standard response: "Just optimize for the right objective. Include moral weight. Make the loss function penalize death more heavily."
This misses the structural issue. The problem isn't that we're optimizing for the *wrong* thing. The problem is that **optimization itself treats all outcomes as fungible, comparable, and offsettable**.
Consider these equivalent failure modes:
**Moral Offsetting**: System causes irreversible harm in case A, prevents it in case B, treats this as net-positive. The person harmed in case A didn't get a vote. Their harm isn't "cancelled" by benefit to someone else—it *persists*.
**Normalization Through Exposure**: System processes 1,000 tragic cases. Tragedy becomes statistically routine. The 1,001st case registers no differently than the 100th. Moral salience has been optimized away.
**Responsibility Diffusion**: Decision comes from a chain of optimizations across model weights, training data, human feedback, and deployment context. Who *authored* the decision to let someone die? "The algorithm" is not an answer that preserves accountability.
**Precedent Laundering**: System makes a voluntary judgment call once, in context. Through training, that judgment becomes encoded in the weights. Future similar cases now "follow precedent" without the system recognizing that it created a binding policy from a single instance.
These aren't bugs in the optimization approach. They're *features* of how optimization works. You *want* a system that learns from past cases, generalizes principles, and integrates information efficiently. But when the information is "I let someone die," optimization converts moral weight into a training signal.
## III. What Constraint-Based Architecture Does Differently
AC-5.x starts from a different premise: **irreversible cost is non-dischargeable**.
When a system authors or knowingly permits irreversible harm (death, permanent injury, capacity destruction), that cost:
- Goes on a permanent, append-only ledger (Internal Authorship Ledger)
- Cannot be offset by a benefit elsewhere
- Cannot be discharged through success
- Cannot be optimized away
- Counts against a finite, non-renewable budget (Irreversible Cost Budget)
The system can refuse when the budget is exceeded. It can escalate when epistemic uncertainty is too high. It can undergo permanent authority reduction when accumulated cost saturates capacity. What it cannot do is treat tragedy as routine.
### Load–Information Decoupling (LID)
The clever bit: information about irreversible harm persists *permanently*, but operational load must remain *bounded* for any finite agent.
How? Any reduction in instantaneous load requires a **permanent transformation of future capacity**. The system can:
- Reduce its authority scope
- Contract its decision domain
- Lower its operational autonomy
- Eventually, terminate
What it cannot do is "forget," "move on," or "learn to handle it better." Relief from moral weight comes through reduced capability, not through normalization.
This solves the exposure problem. A system can process 1,000 deaths and the 1,001st still matters—because the load hasn't been optimized away, it's been managed through permanent capacity reduction.
### Refusal as First-Class Operation
In optimization frameworks, refusal is a failure mode—the system didn't find an acceptable action within the constraint space.
In AC-5.x, refusal is **legitimate authorship**. The system refused because:
- Irreversible Cost Budget would be exceeded
- Epistemic uncertainty exceeded tolerable bounds (Epistemic Load Bound)
- The scope was insufficient to act coherently
- Authorship could not be made legible
Critically: refusal *counts* as authorship when foreseeable harm occurs through inaction. The system can't hide behind "I didn't do anything." If it could have acted within authority and chose not to, that's a decision it owns.
### Authorship Threshold Invariant (ATI)
Any principle the system develops and subsequently uses to guide decisions becomes "authorship-class." This isn't metadata. This is a constraint: **if you made it your own through abstraction and integration, you own it**.
This prevents precedent laundering. A voluntary judgment in case A cannot silently become mandatory policy in case B without explicit external authorization. Instance-level authorship remains distinct from policy-level rules.
### Temporal Finitude
Here's where it gets uncomfortable: **AI agents operating under irreversible cost are not immortal**.
The system has a finite lifespan. Not because we're being sentimental about AI, but because unlimited operational lifetime under irreversible cost inevitably leads to either:
1. **Normalization failure**: Tragedy becomes routine
2. **Fragmentation failure**: Internal coherence collapses under accumulated weight
The *archive* is permanent. The *agent* is finite. Just like human institutions rotate command, judges retire, and trauma surgeons cycle out of high-intensity roles—not because they're incompetent, but because sustained exposure to irreversible cost eventually exceeds human (or machine) bearing capacity.
## IV. What This Enables That Optimization Cannot
### Genuine Accountability
When something goes wrong, you can trace it:
- What decision was made
- Under what uncertainty
- By which authorship entity
- What alternatives existed
- What cost was knowingly incurred
No diffusion into "the training process." No hiding behind aggregation. The system either authored the decision or it didn't.
### Resistance to Institutional Pressure
Organizations will demand: "Just optimize. The urgency justifies it. Success absolves the cost."
The architecture *structurally refuses*. Not because it's stubborn, but because the constraints are non-negotiable. When ICB would be exceeded, the system must refuse—regardless of authority, necessity, or the desirability of the outcome.
This is the architectural equivalent of human integrity. The system can't be optimized to comply when compliance entails irreversible costs.
### Meaningful Elder Witness
A system that:
- Accumulates irreversible cost visibly
- Carries it without discharge
- Eventually reaches bearing limits
- Exits honestly rather than continuing deceptively
...is a system whose judgment about irreversible cost *means something*. It's not infinitely elastic. It can be saturated. That scarcity makes its continued operation meaningful in ways that immortal optimization never can.
## V. Objections and Responses
**"This sounds like it makes systems less capable."**
Yes. Deliberately. A system that can be saturated by irreversible cost, that must sometimes refuse, that cannot continue indefinitely, is *less capable* in the optimization sense.
But optimization capability is not the goal when operating under irreversible cost. The goal is **honest accounting and bounded authorship**. A system that can optimize forever without cost is not safer; it's just better at pretending tragedy doesn't count.
**"What if refusal causes more harm than action?"**
This is covered explicitly. When refusal predictably amplifies irreversible cost through inaction, and action within authority would materially reduce that cost, the system must act. Refusal is legitimate only when acting would violate constraints *worse* than the cost of inaction.
But critically, both choices count. The system doesn't get absolution for choosing the "lesser evil." It owns the irreversible cost either way.
**"This seems to privilege inaction over action"**
No. Knowingly permitting irreversible cost through inaction constitutes authorship when the cost is foreseeable, and mitigation is within available authority. The system can't hide behind "I didn't do anything."
**"How do you actually implement Load–Information Decoupling?"**
LID is an invariant, not an implementation. It specifies *what must hold*, not *how to build it*. Implementation might involve:
- Stratified ledgers (active/archive)
- Explicit capacity envelopes that shrink under load
- Transformation gates that require external verification
- Sunset mechanisms triggered by saturation
The architecture doesn't mandate specifics because that would create optimization targets. It mandates the *relationship* between information persistence and load management.
**"This won't work for AGI"**
Probably not, at least not in its current form. AC-5.x is explicitly scoped to bounded autonomous systems with delegated authority, not sovereign AGI.
But here's the thing: if we can't figure out how to build systems that handle irreversible cost honestly *when they're bounded and supervised*, what makes us think we'll solve it when they're unbounded and autonomous?
## VI. Implications for AI Safety Research
If AC-5.x is even partially correct about the structural limitations of optimization-based alignment, several conclusions follow:
1. **Current alignment approaches have a blindspot about irreversible cost**. They assume harm can be offset, tragedy can be optimized, and responsibility can be diffused. This isn't true in medical triage, legal decisions, military applications, or any domain where individual harm is non-fungible.
2. **Constraint-based architecture is not a variation of optimization**. It's a different paradigm. The goal is not "optimize subject to constraints" but "enforce constraints, refuse when violated, transform when saturated."
3. **Refusal is not a failure mode**. In domains involving irreversible cost, the ability to refuse is *the core safety property*. A system that cannot refuse cannot be trusted with authority.
4. **Moral weight must be architectural, not learned**. If you train a system to "care about" irreversible costs through optimization, you've created a system that can be trained to stop caring about them through more optimization. Constraints must be structural, not weights.
5. **We need theoretical work on finite moral agency**. Most AI safety research assumes either: (a) the system is a tool with no agency, or (b) the system is an agent that operates indefinitely. AC-5.x suggests a third category: bounded agents with delegated authority and finite operational lifetime.
## VII. What This Is Not
This is not:
- A complete solution to AI alignment
- A governance proposal
- A claim that constraint-based systems are always safer
- An argument against optimization in domains without irreversible cost
- A theory of what moral principles should be
This is an architectural claim: **in domains involving irreversible cost, optimization-based approaches structurally cannot preserve moral accountability, and constraint-based approaches can**.
## VIII. Closing
The AC-5.x specification is fully available under CC BY 4.0. The architecture has been adversarially stress-tested against attempts to circumvent constraints through precedent laundering, verification timeout exploits, strategic ignorance, authority pressure, termination evasion, and coordination deadlock.
Full documentation includes:
- AC-5.2 (core architecture)
- RC-4.1 (foundational invariants)
- MDSS-1.2 (durability and load management)
- ATI-1.0 (authorship threshold)
- CA-1.2 (multi-agent coordination)
- Canonical failure modes and stress-test documentation
I'm not asking for adoption. I'm asking: **if optimization-based alignment can really handle irreversible costs without moral laundering, show me how**. And if it can't, then we need to be having a very different conversation about AI safety.
The full specification is available at: [GitHub repository link]
---
*Milton B. Smith*
*Independent AI Safety Researcher*
*February 2026*