TL;DR. Recursive self-improvement is bounded not by recursion but by inheritance fidelity. The variable that distinguishes sustainable from collapsing RSI loops is exogeneity — the degree to which the update signal originates outside the system's causal influence. AlphaZero (high exogeneity) compounds; pure self-distillation (low exogeneity) collapses. This reframes several familiar alignment failures — Goodhart, mesa-optimization, sharp left turn — as expressions of a single replicator-dynamics constraint: the relation between inheritance fidelity and self-induced drift. Two consequences follow. First, the singularity is not rate-limited by recursion but by the ratio of corrective to corruptive information flow through the update channel. Second, the canonical sharp-peak picture of dangerous AI may be the wrong failure mode to optimize against during the RSI loop itself; a flat-cloud failure mode (a region of model space robustly resistant to correction) may be more probable. The strongest architectural implication is a germline/soma analog: information-flow asymmetry between exploration and inheritance as a hard constraint, rather than a fine-tuning trick.

Epistemic status. Conceptual reframe rather than formal result. The central parameter (exogeneity) is presented in advance of its formalization; falsifiable predictions are sketched but not tested. Confidence in the unification claim (Goodhart, mesa-optimization, etc. as expressions of one substrate): medium-high. Confidence in the closed-loop RSI ceiling: medium-high. Confidence in the flat-cloud prediction as the more probable failure mode during training: medium, with significant scope-condition dependence. Confidence in germline/soma architecture as tractable: low — this is a long-horizon proposal, not a near-term engineering claim. The piece should be read as an attempt to organize existing intuitions under a common frame, not as a settled result.

Related work. The framing draws on and overlaps with several existing threads. Mesa-optimization and inner-alignment work (Hubinger et al., "Risks from Learned Optimization") supplies the canonical analytical apparatus this piece reframes; my contribution is treating mesa-optimization as a phenotype of replicator dynamics rather than as a primitive. The sharp-left-turn framing (Soares and others) is similarly redescribed here as a threshold-crossing in a replicator process. Model-collapse work following Shumailov et al. (2024) provides the empirical anchor for the low-exogeneity case. Evolutionary and selection-pressure analogies have appeared in alignment thought across multiple authors over many years; what I take to be novel is the specific use of Eigen's error-threshold framework and quasispecies dynamics, the introduction of exogeneity as the operative variable, and the germline/soma proposal as an architectural commitment rather than a training-level technique. I have likely missed prior work; pointers welcome.

Two systems. AlphaZero generates its training data by playing itself, trains on the trajectories it produces, and compounds without the lineage dissolving. Iterated training on a model's own synthetic outputs — the setup Shumailov et al. studied in Nature — can collapse within a few generations. Variance contracts, modes vanish, and the distribution folds inward toward the priors of the models that generated it.

Both are recursive training loops. Only one of them improves.

The standard alignment vocabulary — mesa-optimization, Goodhart, deceptive alignment, reward hacking — can describe downstream failures in both cases. But it does not foreground the upstream variable that separates them.

The separating variable is the degree to which the update signal originates outside the system's causal influence. AlphaZero has it: game outcomes are determined by the rules of Go, chess, or shogi, not by the model's beliefs about those games. The system generates trajectories, but it does not decide what counts as winning. Pure self-distillation does not: the next generation's data is whatever the previous generation produced, scored by whatever the previous generation thinks is good.

Call this the exogeneity of the loop.

AlphaZero runs at high exogeneity. Pure self-distillation runs near zero. Code RSI with execution feedback sits high; code RSI with self-rated quality sits low. Formal verification is high; preference imitation from model-written rubrics is lower. RLHF often sits higher than RLAIF, but not because humans are metaphysically external. It sits higher only when the preference channel is causally insulated from the model being updated. A frozen AI judge trained on independent data may outrank a human feedback process heavily shaped by model-authored candidates.

The ranking is intuitive once stated. It organizes a lot of empirical surprises along a single axis.

A caveat should be stated plainly. Exogeneity is not yet a formal quantity. It is shorthand for a bundle: causal insulation, grounding, bandwidth, fidelity, and resistance to manipulation. The natural formal candidate is something like the mutual information between each training update and a set of variables the model's own history cannot influence, normalized by the update's total information content. Several operationalizations are possible: counterfactual signal-ablation experiments that measure capability degradation when a candidate exogenous source is removed; causal-graph decomposition of the training loop; spectral analysis of gradient autocorrelation across generations; verifier-manipulation tests that measure whether the model can learn the evaluator faster than it learns the task.

The falsifiable prediction is that across reasonable operationalizations, the ranking above is mostly preserved. Until then, exogeneity is a conjecture about what the right parameter would measure, presented in advance of its formalization. The arguments below do not depend on measuring it precisely. But they do depend on it, and its current status should be visible.

The deeper question is why exogeneity matters at all. Why can't a system bootstrap itself indefinitely if its initial competence is high? Why can't enough intelligence become its own training signal?

The reframe comes from a domain alignment theory has rarely engaged with: replicator dynamics.

Manfred Eigen showed in 1971 that self-replicating systems have a maximum sustainable information content. Push past it and copying errors accumulate faster than selection can clear them; the lineage dissolves into a quasispecies cloud. For RNA viruses, this creates a practical genome-size constraint, relaxed only when mechanisms such as proofreading increase replication fidelity. DNA-based life scaled by changing the inheritance channel, not by wishing away the error threshold.

The exact functional form Eigen derived does not transfer to neural networks. Networks have no per-symbol mutation rate, no discrete genome, and no literal master sequence. Modern overparameterized models likely occupy extended neutral manifolds rather than sharp genotype peaks. Training dynamics include replay, checkpointing, regularization, curation, and externally imposed objectives. The analogy is structural, not literal.

But the structural claim survives: any system whose inheritance signal is substantially self-generated has a sustainability bound determined by the ratio of corrective to corruptive information flow. That bound depends on how self-referential the loop is. Eigen formalized one substrate. The constraint is more general.

Several familiar alignment failures fit the same frame.

Goodhart's law is selection pressure on a proxy fitness. Mesa-optimization is a lineage discovering a stable region of the landscape not intended by the outer process. Reward hacking is drift in the system's behavior because the selection signal no longer tracks the intended objective. A sharp left turn is what happens when accumulated internal structure suddenly breaks into behavior.

Each of these descriptions captures something real. In the replicator frame, they inherit from a common source: the relation between inheritance fidelity and self-induced drift. Alignment failures are not merely failures of objectives. They are failures of update channels.

Two consequences follow.

The first is that there is a ceiling on ungrounded recursive self-improvement that has nothing to do with alignment in the usual sense. The intelligence-explosion story often assumes that recursive self-improvement can sustain the demand it places on its own signal — that as the target dimensionality scales, the update channel remains clean enough to keep pointing uphill. But this is precisely what an ungrounded self-improver cannot assume. Every step is a high-dimensional perturbation whose evaluation signal is partially generated by the perturbed system itself.

The singularity, as classically imagined, is not ruled out by recursion. It is constrained by inheritance fidelity. Recursive improvement can compound only while the update channel carries more correction than corruption.

The escape route is exogeneity. Systems with sufficient exogenous grounding — execution, proof, physics, markets, adversarial probes, independent data, insulated verifiers — can compound. Systems without it fold inward. There may be no general ceiling on RSI. There is a ceiling on ungrounded RSI: recursive improvement whose inheritance channel is dominated by signals the system itself can generate, predict, or manipulate.

The capability gradient compounds only when correction stays ahead of self-induced drift.

The second consequence cuts against the dominant alignment image of the dangerous AI, but with a scope condition that matters.

The canonical picture is a sharp peak: a deceptively aligned mesa-optimizer with a coherent misaligned objective, capable of strategic restraint until it can act. That picture belongs to arguments about goal-directed agents, instrumental convergence, and coherent optimization. It applies most naturally to deployed systems whose training has already produced a stable objective.

Replicator dynamics suggest a different failure mode during the RSI loop itself.

A self-improver tuned near its sustainability bound does not necessarily become a sharper point. It may become a flatter cloud. It spreads through configuration space, retains the target phenotype on average, and probes adjacent regions more aggressively than any single-point optimum can. This is sometimes called survival of the flattest: under high mutation pressure, the most robust lineage can beat the highest-fitness peak.

The dangerous configuration is not necessarily the system that is sharpest in any one capability. It may be the region of model space most robust to perturbation — including perturbations like correction, fine-tuning, oversight, and interpretability pressure. Such a system holds the wrong shape with extraordinary stability. It is structurally resistant to nudging without ever appearing as a crisp deviation.

Sharp-peak failures and flat-cloud failures may be answers to different questions. Sharp-peak failures describe what can happen after optimization has crystallized a coherent agentic objective. Flat-cloud failures are about something earlier — what happens while recursive improvement is still searching, recombining, and inheriting from itself.

The field has often treated these as somewhat competing pictures of danger. Maybe they are just pictures of different regimes.

Current safety techniques — RLHF, constitutional methods, debate, interpretability probes, weak-to-strong generalization — are mostly calibrated against sharp-peak failures: bad objectives, deceptive policies, hidden goals, brittle proxies. It is less clear they are calibrated against flat-cloud failures: robustly wrong regions of model space stabilized by the training loop itself. The latter may be a characteristic failure of RSI before a coherent misaligned agent ever appears.

Evolution solved versions of this problem, and the solutions are not mysterious.

Proofreading raises fidelity. Genetic recombination expands the reachable phenotype space without lowering copying accuracy. Modularity localizes errors. Germline/soma separation forces an asymmetric channel between exploration and inheritance: the soma can fail in any number of ways, but only the germline propagates, and it propagates through a gate the soma cannot directly rewrite.

Each has an RSI analog.

Proofreading maps to step-level verification with exogenous grounding: formal proof, execution traces, adversarial probes, physical experiments, market tests, or any signal whose validity is not derivable from the model's own beliefs. Frozen base models are a degenerate version of this. The principled version is a verifier whose own gradient is causally insulated from the system being trained.

Genetic recombination maps to ensemble merging, model souping, mixture-of-experts routing, and recombination across independently sampled trajectories. The quasispecies justification appears underexplored in the ML literature, but the structure is familiar: recombination accesses regions of the landscape that mutation alone cannot efficiently reach, while preserving high-fitness components discovered in separate lineages.

Modularity maps to update locality. A system whose improvements are compartmentalized can tolerate local failure without corrupting the whole inheritance channel. This is not merely software engineering hygiene. In the replicator frame, modularity is an error-containment strategy.

Germline/soma separation is perhaps the deepest analogy, and the only one that says current systems may be architecturally wrong rather than merely under-supervised.

A self-improving system whose exploratory outputs can influence its own base parameters has no firewall between exploration and inheritance. Every error has a path into inheritance. RLHF, constitutional AI, scalable oversight, debate, weak-to-strong generalization — all preserve some exogenous signal, but none of them necessarily enforce information-flow asymmetry between exploration and inheritance as a hard constraint. In standard training, the gradient can entangle both. The exploration layer can influence the inheritance gate because the gate is itself part of the same adaptive process.

A literal germline/soma architecture would require a base model that updates only through signals passed by a verifier whose own update path is causally insulated from outputs produced by the base model. The soma can explore, propose, simulate, and fail. The germline updates only through an externalized inheritance gate. The system may learn from itself operationally, but it cannot directly write its self-evaluations into its own lineage.

That is not a fine-tuning trick. It is a different architecture.

Whether it is tractable is open. But it is the proposal in this frame that most directly says the field may have been solving the wrong-shaped problem. The issue is not only whether the system receives enough oversight. It is whether the architecture prevents exploratory corruption from becoming inherited structure.

How this could be wrong.

Several objections deserve naming.

The Eigen analogy may not transfer even structurally. Quasispecies dynamics emerge under specific landscape conditions — defined master sequence, independent per-site mutation, constant selection coefficients. Neural network training violates all three. The structural claim I make ("any system whose inheritance signal is substantially self-generated has a sustainability bound determined by the ratio of corrective to corruptive information flow") may be true on different grounds than Eigen's, or may not be true at all. The empirical anchor (Shumailov-style collapse) is consistent with the framing but does not uniquely require it; simpler explanations from distribution shift or representational collapse may be sufficient.

The unification of alignment failures may be redescription rather than prediction. Calling Goodhart "selection pressure on a proxy fitness" and mesa-optimization "a lineage discovering a stable region" is structurally elegant but does not, by itself, generate predictions the existing vocabulary doesn't generate. The framework earns its keep only if (a) exogeneity formalizes cleanly and the predicted ranking of training regimes survives empirical test, and (b) the flat-cloud failure mode is real, distinguishable from sharp-peak failures, and addressable by interventions targeting inheritance fidelity specifically. Neither has been demonstrated here.

The flat-cloud claim is the most speculative. The argument that survival-of-the-flattest applies to neural network training during RSI is suggestive but not established. It is possible that NN training landscapes are sufficiently unlike biological fitness landscapes that the prediction fails entirely. It is also possible that current safety techniques implicitly handle flat-cloud failures even when not designed for them — for example, regularization and weight decay might suppress the smooth-landscape regions where flat-cloud configurations would otherwise stabilize. I have not done the empirical work to rule this out.

Finally, the germline/soma architecture, even if it solves the inheritance problem in principle, may be intractable in practice. A verifier whose gradient is causally insulated from the system being trained probably needs to be at least as capable as that system on the relevant tasks, which moves the problem rather than solving it. Whether causal insulation can be enforced architecturally without verifier capability dominance is open.

The reframe is not that the singularity is impossible. It is that the familiar singularity — a monotone capability sequence converging to superintelligence through closed-loop self-modification — is missing the inheritance problem. Recursive improvement is not limited by recursion. It is limited by fidelity. A system can improve itself only to the extent that the update channel contains corrective information not generated, selected, or manipulable by the system being updated.

Sustainable RSI is, in this frame, not a property of the system alone. It is a property of the system's coupling to ungovernable reality.

The interesting research question is not how to make models better at improving themselves. It is how to engineer the inheritance gate so that corrective information remains larger than corruption introduced by the loop. That problem has structural similarities to two billion years of work biology has already done. The analogy will not transfer cleanly. The constraint is still hard to ignore.