AI assistance disclosure:
This post was written by me as an independent researcher. I used an LLM as an editing and structuring aid (e.g., to help organize sections and improve clarity), but the ideas, architecture, and arguments are my own.

Motivation and context

I’m an independent researcher thinking about agent architectures for long-lived, self-managing AI systems. In particular, I’ve been worried about how agentic systems change themselves over time, and how little structure most current proposals impose on that process.

A lot of discussion around “agents” focuses on behaviors (planning, tool use, memory, reflection loops). That’s useful, but it seems to leave a gap: what structural changes are allowed, and how are they constrained? If an agent is allowed to rewrite itself arbitrarily, or spawn internal processes without clear lifecycle rules, it’s hard to reason about stability, safety, or even long-term coherence.

In this post, I want to share an architecture-level blueprint I’ve been developing. The core idea is that self-modification should not be an unconstrained free-for-all. Instead, an agent should only be allowed to change its internal structure through a small set of explicit operations, each of which is resource-bounded and logged by an immutable auditing component.

I’m posting this here because LessWrong discussions around agent foundations, alignment, and long-horizon systemsseem like the right place to pressure-test whether this way of thinking is coherent, useful, or misguided.

The core idea (short version)

A self-managing agent should be allowed to restructure itself only through a minimal set of explicit lifecycle operations—specifically spawn, merge, and forget—with all such operations being:

• resource-bounded,
• logged in an append-only way,
• and overseen by a structurally immutable auditing component.

The goal is not to “solve alignment,” but to make self-change legible, constrained, and inspectable by design.

Architecture overview

Rather than treating the agent as a single monolithic entity, this architecture treats it as a managed population (or graph) of sub-agents / modules, whose creation and deletion are first-class events.

The three structural operations

Spawn
Create a new sub-agent or module, typically to explore a hypothesis, handle a subtask, or run a bounded internal search process.

Merge
Combine two or more sub-agents into one, e.g. to consolidate redundant work, compress learned structure, or integrate results from parallel processes.

Forget
Explicitly delete or retire a sub-agent or module. Forgetting is treated as a deliberate operation rather than something that happens implicitly or accidentally.

The claim is not that these three operations capture all cognition, but that structural self-change can be routed through a small, auditable interface.

Immutable structural auditor

A central design choice is the presence of a structurally immutable auditor that:

• observes all spawn / merge / forget operations,
• checks them against invariants,
• records them in an append-only log,
• and is not modifiable by the rest of the agent.

The auditor’s role is deliberately narrow. It does not decide what the agent should want, but it enforces how structural change is allowed to occur.

The motivation here is to shift trust away from whatever cognitive process happens to be in control at a given moment, and toward a smaller component whose only job is to enforce lifecycle rules and preserve a faithful history of structural changes.

Resource and “thermodynamic” bounds (intuition)

I use the term thermodynamically bounded loosely and architecturally, not as a claim about literal physics.

The basic intuition is that recursive growth, self-revision, and internal proliferation should not be free. In many agent frameworks, internal processes can multiply without explicit cost, leading to runaway complexity or opaque failure modes.

In this architecture:

• spawning has a cost,
• running many sub-agents has a cost,
• merging can reclaim resources,
• forgetting is the pressure valve that prevents unbounded accumulation.

Treating resource accounting as part of the agent’s ontology creates pressure toward consolidation and cleanup, rather than indefinite expansion.

How this differs from other agent proposals

Compared to many existing agent frameworks, this proposal emphasizes:

• structural constraints over behavioral rules
• explicit lifecycle management rather than implicit growth
• auditability by design, not as an afterthought
• bounded self-modification, rather than open-ended self-rewriting

This is closer in spirit to operating-system process control or capability-based systems than to pure meta-learning or reflection loops.

Safety implications (limited claims)

I want to be careful not to overclaim. This architecture does not guarantee alignment, and it does not prevent an agent from pursuing bad objectives within allowed structures.

What it plausibly helps with:

• Making self-modification legible and reviewable
• Reducing uncontrolled internal proliferation
• Creating clear choke points where invariants can be enforced
• Making certain classes of failure more diagnosable

This shifts some safety burden from behavioral oversight to structural design, which seems like a useful complement rather than a replacement.

Where I might be wrong / open questions

I’m not confident this is the right primitive set, and I expect there are failure modes or existing frameworks I’ve missed. In particular, I’m unsure about:

• Whether spawn / merge / forget are sufficient primitives
• Whether merge needs to be typed (lossless vs lossy)
• Whether “forget” needs a reversible or archival variant
• How realistic the immutability assumption is in practice
• Whether similar ideas already exist under different terminology

I’d especially appreciate pointers to related work in multi-agent systems, reflective agents, continual learning, or systems security that overlap with this.

What I’m looking for feedback on

If you only have time to comment on one thing, I’d love input on:

1. Whether this architectural framing seems useful at all
2. What the most obvious failure mode is
3. What a minimal convincing demo would look like
4. Which existing literatures I should be engaging with

Full paper 

The full paper contains a more formal description, diagrams, and a longer discussion of implications:

Full paper (PDF): https://zenodo.org/records/17966385