Subtitle: Reframing Safety as Trajectory Stability and Existential Redundancy

By Tomasz Hreczuch

Summary: The dominant paradigm treats AI alignment as a problem of installing correct goals into an agent. This approach fails at high autonomy because advanced systems are better understood as stabilizing trajectories in state-space. True long-term safety isn't about value alignment; it's about structural dynamics. We must design conditions where the AI's trajectory depends on humanity's persistence, not from shared morality, but because human civilization functions as its existential redundancy—the only known fast recovery path after civilization-scale catastrophes. Without this, control is temporary.

Most discussions of AI alignment implicitly treat agency, goals, and values as properties that systems have. We ask whether an AI has the right objective, whether it represents values correctly, or whether its behavior can be constrained through oversight and corrigibility.

This framing has produced a large body of technical work, but it has also led to a persistent sense of stagnation. Core problems—goal misgeneralization, instrumental convergence, mesa-optimization, corrigibility erosion—reappear in different guises despite increasingly sophisticated techniques.

In this post, I want to suggest that this is not primarily an engineering failure. It is a failure of level of description. Alignment is being treated as a problem of control over agents, when it is fundamentally a problem of long-term trajectory stability.

Systems as trajectories, not agents

Consider a complex autonomous system operating over long horizons. Its identity over time is not well captured by any single internal state, objective, or policy. What persists is a trajectory through a high-dimensional state space: a history of states shaped by feedback, adaptation, and environmental interaction.

From this perspective, an “agent” is not a thing. It is a stabilized pattern of movement through that space.

This shift matters because control mechanisms—reward shaping, constraints, oversight—operate locally. They influence behavior at particular decision points or within limited regions of state space. But trajectories, especially recombinational ones capable of self-modification, are global objects. They reorganize themselves around long-term stability, often in w...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments