This is an automated rejection. No LLM generated, heavily assisted/co-written, or otherwise reliant work.
Read full explanation
From Endless Loops to Hierarchical Thought
This post argues for a specific architectural claim about agentic systems built on frontier LLMs: that the dominant “sequential loop” pattern—accumulating an ever-growing transcript and asking the model what to do next—hits a scaling wall that cannot be solved simply by larger context windows, better summarisation or even better memory models.
I propose that longer-horizon, interdependent tasks require explicit hierarchical decomposition plus an executable planning representation, and—crucially—a repair mechanism that treats plan structure as a provisional hypothesis rather than a brittle contract.
I’m posting here because this intersects with LessWrong’s interests in epistemic hygiene, failure modes of cognition under noise, and the design of systems that can be inspected, corrected, and updated under uncertainty. I would particularly value pushback on: (i) whether the “transcript is not memory” diagnosis is right, (ii) whether repair is truly structural rather than an implementation detail, and (iii) what counterexamples exist where sequential loops scale without silently accumulating incoherence.
I wrote this piece and used an LLM only for light editing (phrasing/structure). The arguments and claims are mine, and I’m happy to clarify or revise them under critique.
Claims
Core claims
The transcript-shaped sequential loop cannot scale to deep, long-horizon work. As iterations accumulate, the context becomes a mixture of revisions, retractions, and superseded assumptions; signal-to-noise degrades, stale premises resurface, and cost/latency rise monotonically. Summarisation helps but introduces irreversible loss and flattens epistemic distinctions. Eloquent Kitsune
Decomposition is not optional; it is a structural constraint. Systems that solve large problems must operate with locality, bounded inputs/outputs, and insulated subcontexts. For agents, this entails explicit decomposition into subgoals and dependencies rather than monolithic “global context.” Eloquent Kitsune
Explicit planning requires a design language, and that introduces brittleness. Executable plans force names, scopes, IO contracts, dependency rules, and failure representations. Many systems avoid this because when structure breaks, failure is abrupt (schema violations, invalid plans, misrouting). Eloquent Kitsune
Therefore, repair is unavoidable and must be first-class. The resolution is to treat structure as a provisional hypothesis about the problem: execution reveals mismatches; mismatches should trigger plan repair rather than collapse back into free-form prompting. Without repair, structured decomposition is unusable under uncertainty; with repair, it becomes scalable. Eloquent Kitsune
The deeper advantage is semantic isolation, not merely token efficiency. Hierarchies constrain which information can interact; only selected outputs propagate. This contains stale premises, localises revisions, and makes coupling explicit rather than accidental—something sequential prompting cannot reliably achieve regardless of context size. Eloquent Kitsune
Falsifiable implications / what would change my mind (this reads very “LW” and lowers moderation risk)
If you can show a sequential-loop agent that (a) sustains long-horizon coherence, (b) prevents premise leakage without explicit structure, and (c) does so without relying on hidden structured state outside the transcript, that would weaken Claim 1.
If repair can be removed entirely (no plan revision under mismatch) while retaining the benefits of decomposition across diverse tasks, that would weaken Claim 4
Preface
Modern frontier language models have strong reasoning capabilities. They can plan, revise, self-correct, and sustain non-trivial chains of thought. Yet despite this progress at the model level, most agentic systems built on top of them converge on a simple architectural pattern.
For most developers, the default guidance is unchanged: implement a recursive loop that prepends prior context, asks the model what to do next, executes the result, and repeats.
This pattern is an extension of chat-based interaction. A human adds context incrementally; the model responds. As systems became autonomous, the human was removed, but the conversational structure remained. The loop persisted.
As a result, much of today’s agent infrastructure is still conversational in shape, even when applied to tasks that are no longer conversational.
This architecture persists because it is easy to build, requires no explicit schema or planning language, operates entirely in free-form text, and—crucially—fails softly. When it degrades, it drifts or repeats rather than crashing outright.
Structured systems do not fail this way. When structure breaks, failure is often abrupt: invalid plans, schema violations, misrouted execution. Faced with this brittleness, many systems retreat to ever-growing prompts.
But this approach does not scale. As agents are asked to solve longer-horizon, interdependent problems, its limitations become unavoidable.
1. Why the Sequential Loop Cannot Scale
The sequential loop feels robust because nothing is explicitly discarded. The system always “has the full history.” In principle, it can refer back to any prior assumption or correction.
Here is the shape of the traditional agentic loop most systems still implement:
Traditional agentic loop (sequential, transcript-shaped context).
In practice, this produces noise.
As conversations progress, they accumulate revisions, partial retractions, deprecated assumptions, and superseded plans. Earlier misunderstandings are rarely removed; they are overwritten conversationally. All versions coexist in the prompt.
The model is expected to infer which parts of the history still matter.
Most of the time, this works. Recency bias helps. Conversational cues signal that earlier assumptions are invalid. The model usually selects the latest interpretation.
But not always.
Outdated premises resurface. Abandoned constraints reappear. Each turn adds not just information, but interference. The signal-to-noise ratio degrades steadily.
This compounds with scale. Token growth is superlinear in problem complexity. Latency and cost rise monotonically. Summarisation mitigates this but introduces irreversible information loss.
More subtly, sequential prompting assigns equal representational weight to everything that survives summarisation. Old and new assumptions share the same textual surface. The system resolves epistemic conflicts implicitly, rather than structurally.
This is not a memory system. It is a transcript. And transcripts are a poor substrate for sustained reasoning.
2. Decomposition Is Not Optional
Any system attempting to solve sufficiently large, goal-directed problems must eventually adopt explicit decomposition.
This is not a matter of taste. It is a structural constraint that appears across domains. Human cognition relies on bounded attention. Software systems decompose because monoliths collapse. Mathematics advances through lemmas, not uninterrupted inference.
Global context is a convenience abstraction, not a workable one.
As problem complexity grows, context locality becomes mandatory. Subsystems must operate on bounded inputs, produce scoped outputs, and remain insulated from irrelevant information.
In agentic systems, decomposition naturally implies plans.
A plan is not merely a list of steps. It is an explicit representation of problem structure: subgoals, dependencies, execution order, and information flow. It externalizes reasoning so it can be inspected, executed, and revised.
Hierarchical reasoning replaces monolithic context with structured problem graphs, where each node reasons locally.
Hierarchical decomposition is often imbalanced: some sub-goals are terminal, others decompose further.
This shift is not primarily about efficiency. It is about making deep reasoning possible at all.
3. The Real Difficulty: Committing to Structure Under Uncertainty
Decomposition requires commitment to structure.
Structure introduces brittleness.
To decompose a task, a system must decide what the parts are, where boundaries lie, what information flows between them, and what counts as success. These are acts of compression performed under uncertainty.
This is where many systems fail.
Plans require names, scopes, inputs, and outputs. To be executable, they must obey a design language governing composition and dependency. Without this, plans collapse back into prose.
But committing to structure too early risks error. Boundaries may be wrong. Dependencies misunderstood. Execution may reveal that the problem does not factor cleanly.
When systems treat these failures as fatal, decomposition becomes unusable. Schema violations halt execution. Incorrect plans are discarded rather than revised.
This creates a dilemma:
Unstructured systems are robust but do not scale.
Structured systems scale but are brittle.
Resolving this tension requires rethinking the relationship between structure and correctness.
4. From Decomposition to Structured Plans — and Why Repair Is Unavoidable
Once a system commits to decomposition, that decomposition must be represented explicitly.
A hierarchy of subproblems cannot remain implicit if it is to be reasoned about, executed, or revised. It must be rendered into a structured plan. Without this, there is nothing to inspect or manipulate.
But the moment a plan exists, a second constraint appears.
To be executable, a plan must obey a design language. There must be rules governing how steps relate, how outputs connect to inputs, how failure is represented, and how progress is tracked.
This exposes assumptions.
Plans encode beliefs about the problem that may be incomplete or wrong. Execution surfaces mismatches. If these are treated as terminal failures, structure becomes a liability.
The resolution is not weaker structure, but different failure handling.
A structured plan must be treated as a provisional hypothesis about problem structure. It must be expected to be wrong in places. Therefore, it must be repairable.
Repair is not an implementation detail. It is a structural requirement. Without it, explicit planning collapses under uncertainty. With it, structure becomes negotiable.
A repair loop turns planning into an iterative process:
Repair makes structure revisable instead of brittle.
Failure becomes information about the adequacy of the structure itself.
Only under this framing does explicit decomposition scale.
5. Why This Actually Scales: From Token Efficiency to Semantic Isolation
Hierarchical decomposition with repair yields computational benefits. Reasoning occurs over bounded contexts. Token usage drops. Latency improves. Subproblems can be recomputed independently.
But these gains are not the primary reason this approach scales.
The deeper advantage is semantic isolation.
In sequential loops, all prior assumptions remain co-present. The model must sort relevance heuristically under noise. Errors propagate because nothing enforces separation.
In decomposed systems, reasoning paths are structurally isolated. Each subproblem operates within a constrained epistemic boundary. Only explicitly selected outputs propagate forward.
This changes system behavior fundamentally:
Stale premises are contained.
Revisions are localized.
Coupling is explicit rather than accidental.
This is how large software systems remain intelligible. Complexity is not eliminated; it is partitioned.
Sequential prompting cannot achieve this, regardless of context size. Without explicit structure, relevance remains heuristic and leakage unavoidable.
6. The Philosophical Shift: Intelligence Through Controlled Interaction
This architectural shift implies a deeper correction.
A common assumption in LLM tooling is that intelligence scales with larger contexts. If the system could only remember more, it would reason better.
But accumulation is not understanding.
Intelligence emerges from controlled interaction between relevant parts of a problem. Memory without structure produces noise. Attention without boundaries produces interference.
Hierarchical decomposition with repair asserts a different principle: reasoning fidelity depends not on how much information is retained, but on how rigorously interaction is governed.
Forgetting is not a failure mode. It is a prerequisite for coherence.
7. A Blueprint for Reasoning Fidelity Under Decomposition
This essay does not advocate a specific framework or implementation. It outlines a blueprint for maintaining reasoning fidelity as systems decompose problems into parts.
The hierarchy enforces locality.
The planning language externalizes structure.
The repair mechanism allows that structure to evolve without collapse.
Together, these enable systems to scale in depth, not just surface fluency.
8. What This Unlocks
As agentic systems grow more capable, expectations will rise. We will ask them to solve longer-horizon problems, manage interdependent constraints, and adapt to evolving goals.
Without decomposition, this produces increasingly confident incoherence.
With hierarchical structure and repair, it becomes possible to build systems that fail softly without abandoning structure, that forget intentionally, and that sustain coherent reasoning under complexity.
The era of endless prompt accumulation is ending. The next generation of systems will not be defined by how much they remember, but by how well they decide what no longer matters.
Provenance This post was originally published on my personal site: The Next Breakpoint in LLM Design: From Endless Loops to Hierarchical Thought — https://timgee.co.uk/essays/next-breakpoint/
From Endless Loops to Hierarchical Thought
This post argues for a specific architectural claim about agentic systems built on frontier LLMs: that the dominant “sequential loop” pattern—accumulating an ever-growing transcript and asking the model what to do next—hits a scaling wall that cannot be solved simply by larger context windows, better summarisation or even better memory models.
I propose that longer-horizon, interdependent tasks require explicit hierarchical decomposition plus an executable planning representation, and—crucially—a repair mechanism that treats plan structure as a provisional hypothesis rather than a brittle contract.
I’m posting here because this intersects with LessWrong’s interests in epistemic hygiene, failure modes of cognition under noise, and the design of systems that can be inspected, corrected, and updated under uncertainty. I would particularly value pushback on: (i) whether the “transcript is not memory” diagnosis is right, (ii) whether repair is truly structural rather than an implementation detail, and (iii) what counterexamples exist where sequential loops scale without silently accumulating incoherence.
I wrote this piece and used an LLM only for light editing (phrasing/structure). The arguments and claims are mine, and I’m happy to clarify or revise them under critique.
Claims
Core claims
As iterations accumulate, the context becomes a mixture of revisions, retractions, and superseded assumptions; signal-to-noise degrades, stale premises resurface, and cost/latency rise monotonically. Summarisation helps but introduces irreversible loss and flattens epistemic distinctions. Eloquent Kitsune
Systems that solve large problems must operate with locality, bounded inputs/outputs, and insulated subcontexts. For agents, this entails explicit decomposition into subgoals and dependencies rather than monolithic “global context.” Eloquent Kitsune
Executable plans force names, scopes, IO contracts, dependency rules, and failure representations. Many systems avoid this because when structure breaks, failure is abrupt (schema violations, invalid plans, misrouting). Eloquent Kitsune
The resolution is to treat structure as a provisional hypothesis about the problem: execution reveals mismatches; mismatches should trigger plan repair rather than collapse back into free-form prompting. Without repair, structured decomposition is unusable under uncertainty; with repair, it becomes scalable. Eloquent Kitsune
Hierarchies constrain which information can interact; only selected outputs propagate. This contains stale premises, localises revisions, and makes coupling explicit rather than accidental—something sequential prompting cannot reliably achieve regardless of context size. Eloquent Kitsune
Falsifiable implications / what would change my mind (this reads very “LW” and lowers moderation risk)
Preface
Modern frontier language models have strong reasoning capabilities. They can plan, revise, self-correct, and sustain non-trivial chains of thought. Yet despite this progress at the model level, most agentic systems built on top of them converge on a simple architectural pattern.
For most developers, the default guidance is unchanged: implement a recursive loop that prepends prior context, asks the model what to do next, executes the result, and repeats.
This pattern is an extension of chat-based interaction. A human adds context incrementally; the model responds. As systems became autonomous, the human was removed, but the conversational structure remained. The loop persisted.
As a result, much of today’s agent infrastructure is still conversational in shape, even when applied to tasks that are no longer conversational.
This architecture persists because it is easy to build, requires no explicit schema or planning language, operates entirely in free-form text, and—crucially—fails softly. When it degrades, it drifts or repeats rather than crashing outright.
Structured systems do not fail this way. When structure breaks, failure is often abrupt: invalid plans, schema violations, misrouted execution. Faced with this brittleness, many systems retreat to ever-growing prompts.
But this approach does not scale. As agents are asked to solve longer-horizon, interdependent problems, its limitations become unavoidable.
1. Why the Sequential Loop Cannot Scale
The sequential loop feels robust because nothing is explicitly discarded. The system always “has the full history.” In principle, it can refer back to any prior assumption or correction.
Here is the shape of the traditional agentic loop most systems still implement:
Traditional agentic loop (sequential, transcript-shaped context).
In practice, this produces noise.
As conversations progress, they accumulate revisions, partial retractions, deprecated assumptions, and superseded plans. Earlier misunderstandings are rarely removed; they are overwritten conversationally. All versions coexist in the prompt.
The model is expected to infer which parts of the history still matter.
Most of the time, this works. Recency bias helps. Conversational cues signal that earlier assumptions are invalid. The model usually selects the latest interpretation.
But not always.
Outdated premises resurface. Abandoned constraints reappear. Each turn adds not just information, but interference. The signal-to-noise ratio degrades steadily.
This compounds with scale. Token growth is superlinear in problem complexity. Latency and cost rise monotonically. Summarisation mitigates this but introduces irreversible information loss.
More subtly, sequential prompting assigns equal representational weight to everything that survives summarisation. Old and new assumptions share the same textual surface. The system resolves epistemic conflicts implicitly, rather than structurally.
This is not a memory system. It is a transcript. And transcripts are a poor substrate for sustained reasoning.
2. Decomposition Is Not Optional
Any system attempting to solve sufficiently large, goal-directed problems must eventually adopt explicit decomposition.
This is not a matter of taste. It is a structural constraint that appears across domains. Human cognition relies on bounded attention. Software systems decompose because monoliths collapse. Mathematics advances through lemmas, not uninterrupted inference.
Global context is a convenience abstraction, not a workable one.
As problem complexity grows, context locality becomes mandatory. Subsystems must operate on bounded inputs, produce scoped outputs, and remain insulated from irrelevant information.
In agentic systems, decomposition naturally implies plans.
A plan is not merely a list of steps. It is an explicit representation of problem structure: subgoals, dependencies, execution order, and information flow. It externalizes reasoning so it can be inspected, executed, and revised.
Hierarchical reasoning replaces monolithic context with structured problem graphs, where each node reasons locally.
Hierarchical decomposition is often imbalanced: some sub-goals are terminal, others decompose further.
This shift is not primarily about efficiency. It is about making deep reasoning possible at all.
3. The Real Difficulty: Committing to Structure Under Uncertainty
Decomposition requires commitment to structure.
Structure introduces brittleness.
To decompose a task, a system must decide what the parts are, where boundaries lie, what information flows between them, and what counts as success. These are acts of compression performed under uncertainty.
This is where many systems fail.
Plans require names, scopes, inputs, and outputs. To be executable, they must obey a design language governing composition and dependency. Without this, plans collapse back into prose.
But committing to structure too early risks error. Boundaries may be wrong. Dependencies misunderstood. Execution may reveal that the problem does not factor cleanly.
When systems treat these failures as fatal, decomposition becomes unusable. Schema violations halt execution. Incorrect plans are discarded rather than revised.
This creates a dilemma:
Resolving this tension requires rethinking the relationship between structure and correctness.
4. From Decomposition to Structured Plans — and Why Repair Is Unavoidable
Once a system commits to decomposition, that decomposition must be represented explicitly.
A hierarchy of subproblems cannot remain implicit if it is to be reasoned about, executed, or revised. It must be rendered into a structured plan. Without this, there is nothing to inspect or manipulate.
But the moment a plan exists, a second constraint appears.
To be executable, a plan must obey a design language. There must be rules governing how steps relate, how outputs connect to inputs, how failure is represented, and how progress is tracked.
This exposes assumptions.
Plans encode beliefs about the problem that may be incomplete or wrong. Execution surfaces mismatches. If these are treated as terminal failures, structure becomes a liability.
The resolution is not weaker structure, but different failure handling.
A structured plan must be treated as a provisional hypothesis about problem structure. It must be expected to be wrong in places. Therefore, it must be repairable.
Repair is not an implementation detail. It is a structural requirement. Without it, explicit planning collapses under uncertainty. With it, structure becomes negotiable.
A repair loop turns planning into an iterative process:
Repair makes structure revisable instead of brittle.
Failure becomes information about the adequacy of the structure itself.
Only under this framing does explicit decomposition scale.
5. Why This Actually Scales: From Token Efficiency to Semantic Isolation
Hierarchical decomposition with repair yields computational benefits. Reasoning occurs over bounded contexts. Token usage drops. Latency improves. Subproblems can be recomputed independently.
But these gains are not the primary reason this approach scales.
The deeper advantage is semantic isolation.
In sequential loops, all prior assumptions remain co-present. The model must sort relevance heuristically under noise. Errors propagate because nothing enforces separation.
In decomposed systems, reasoning paths are structurally isolated. Each subproblem operates within a constrained epistemic boundary. Only explicitly selected outputs propagate forward.
This changes system behavior fundamentally:
This is how large software systems remain intelligible. Complexity is not eliminated; it is partitioned.
Sequential prompting cannot achieve this, regardless of context size. Without explicit structure, relevance remains heuristic and leakage unavoidable.
6. The Philosophical Shift: Intelligence Through Controlled Interaction
This architectural shift implies a deeper correction.
A common assumption in LLM tooling is that intelligence scales with larger contexts. If the system could only remember more, it would reason better.
But accumulation is not understanding.
Intelligence emerges from controlled interaction between relevant parts of a problem. Memory without structure produces noise. Attention without boundaries produces interference.
Hierarchical decomposition with repair asserts a different principle: reasoning fidelity depends not on how much information is retained, but on how rigorously interaction is governed.
Forgetting is not a failure mode. It is a prerequisite for coherence.
7. A Blueprint for Reasoning Fidelity Under Decomposition
This essay does not advocate a specific framework or implementation. It outlines a blueprint for maintaining reasoning fidelity as systems decompose problems into parts.
The hierarchy enforces locality.
The planning language externalizes structure.
The repair mechanism allows that structure to evolve without collapse.
Together, these enable systems to scale in depth, not just surface fluency.
8. What This Unlocks
As agentic systems grow more capable, expectations will rise. We will ask them to solve longer-horizon problems, manage interdependent constraints, and adapt to evolving goals.
Without decomposition, this produces increasingly confident incoherence.
With hierarchical structure and repair, it becomes possible to build systems that fail softly without abandoning structure, that forget intentionally, and that sustain coherent reasoning under complexity.
The era of endless prompt accumulation is ending. The next generation of systems will not be defined by how much they remember, but by how well they decide what no longer matters.
Provenance
This post was originally published on my personal site:
The Next Breakpoint in LLM Design: From Endless Loops to Hierarchical Thought — https://timgee.co.uk/essays/next-breakpoint/