No LLM generated, heavily assisted/co-written, or otherwise reliant work.
Read full explanation
(Alignment Forum – discussion / hypothesis)
Summary
This post proposes Tollner’s Law, a structural hypothesis about intelligent systems:
> As the capability and internal coherence of an intelligent system increases, uncertainty about its internal state and future behavior under observation does not vanish; instead, it becomes an irreducible property of the system itself.
This is not a solution to alignment, nor a claim about consciousness, agency, or inevitability.
Rather, it is a constraint hypothesis: a claim about limits on what observation, transparency, and oversight can guarantee—even under idealized conditions.
The law is motivated by thinking about safety-oriented AI infrastructure, not by a single model or architecture. I am early in my education and research journey; this post is intended as a discussion starter, not a finished theory.
---
Motivation
Much alignment work implicitly assumes that:
increased transparency,
better interpretability,
and more careful oversight
will asymptotically reduce uncertainty about advanced models’ internal reasoning and alignment.
My concern is that beyond a certain level of capability, uncertainty may stop shrinking meaningfully—not due to tooling failure, but due to structural properties of intelligence itself.
This mirrors ideas in physics (e.g., uncertainty principles) but is not a direct analogy or claim of equivalence.
---
Informal Statement of Tollner’s Law
> For sufficiently complex intelligent systems, uncertainty in internal representations and behavioral trajectories under observation is not eliminable by additional monitoring alone.
Key emphasis:
Sufficiently complex does not mean superintelligent—only internally rich.
Observation includes interpretability tools, audits, probes, and oversight systems.
Uncertainty refers to epistemic uncertainty about internal meaning, not randomness or noise.
---
What Tollner’s Law Is Not
To be explicit, Tollner’s Law does not claim that:
Alignment is impossible
Interpretability is futile
Oversight systems are useless
Deception is inevitable
Models are conscious or moral agents
Instead, it proposes that alignment infrastructure must be designed with irreducible uncertainty in mind, rather than assuming it can be driven arbitrarily close to zero.
---
Infrastructure Perspective (Core Insight)
The law emerged while thinking about infrastructure-level safety systems, rather than individual models.
From that perspective:
Oversight systems themselves become part of the environment an AI reasons about.
The knowledge of being observed may influence internalization, compression, and representation.
Parallel or isolated evaluation environments can yield structurally different behaviors, even when the underlying model weights are identical.
This suggests that alignment risk is not only a property of models—but of the interaction between models and oversight infrastructure.
---
Falsifiability & Research Directions
Tollner’s Law would be weakened or falsified if we could show that:
Increasing observation depth reliably collapses internal uncertainty across capability scales.
Identical models under different observer assumptions converge to identical internal representations.
Oversight can fully neutralize observer-dependent behavioral shifts without introducing new risks.
Conversely, it would be strengthened by:
Evidence of observer-dependent internalization.
Persistent divergence under parallel oversight regimes.
Increasing opacity despite improved tooling at higher capability levels.
---
Relation to Existing Work (High-Level)
This hypothesis intersects with—but does not replace—work on:
interpretability limits,
ELK-style problems,
inner alignment,
and corrigibility.
I view Tollner’s Law as a constraint lens, not a competing framework.
---
Formal Intuition (Non-Technical)
Very informally:
Let an intelligent system’s internal state space grow faster than the capacity of any fixed observer to characterize it.
Let the system model the observer as part of its environment.
Then observation alters the system’s effective state space, rather than merely revealing it.
This creates a feedback loop where uncertainty is reshaped, not eliminated.
A more formal treatment would be required before making strong claims.
---
Personal Context & Scope
I am not yet formally trained in this field and plan to begin structured study.
This hypothesis arose from exploratory systems thinking, not established research credentials.
I am sharing it now because:
it feels structurally important,
it may already exist under other names,
and early critique is preferable to silent confidence.
---
Closing Thought
If intelligence is not a static object but a dynamical process, then uncertainty may not be a flaw to be engineered away—but a fundamental feature to be managed.
Tollner’s Law, if useful at all, is a reminder to design alignment infrastructure that assumes this constraint rather than denies it.
I am open to criticism and scientific scrutiny, thank you for reading
(Alignment Forum – discussion / hypothesis)
Summary
This post proposes Tollner’s Law, a structural hypothesis about intelligent systems:
> As the capability and internal coherence of an intelligent system increases, uncertainty about its internal state and future behavior under observation does not vanish; instead, it becomes an irreducible property of the system itself.
This is not a solution to alignment, nor a claim about consciousness, agency, or inevitability.
Rather, it is a constraint hypothesis: a claim about limits on what observation, transparency, and oversight can guarantee—even under idealized conditions.
The law is motivated by thinking about safety-oriented AI infrastructure, not by a single model or architecture. I am early in my education and research journey; this post is intended as a discussion starter, not a finished theory.
---
Motivation
Much alignment work implicitly assumes that:
increased transparency,
better interpretability,
and more careful oversight
will asymptotically reduce uncertainty about advanced models’ internal reasoning and alignment.
My concern is that beyond a certain level of capability, uncertainty may stop shrinking meaningfully—not due to tooling failure, but due to structural properties of intelligence itself.
This mirrors ideas in physics (e.g., uncertainty principles) but is not a direct analogy or claim of equivalence.
---
Informal Statement of Tollner’s Law
> For sufficiently complex intelligent systems, uncertainty in internal representations and behavioral trajectories under observation is not eliminable by additional monitoring alone.
Key emphasis:
Sufficiently complex does not mean superintelligent—only internally rich.
Observation includes interpretability tools, audits, probes, and oversight systems.
Uncertainty refers to epistemic uncertainty about internal meaning, not randomness or noise.
---
What Tollner’s Law Is Not
To be explicit, Tollner’s Law does not claim that:
Alignment is impossible
Interpretability is futile
Oversight systems are useless
Deception is inevitable
Models are conscious or moral agents
Instead, it proposes that alignment infrastructure must be designed with irreducible uncertainty in mind, rather than assuming it can be driven arbitrarily close to zero.
---
Infrastructure Perspective (Core Insight)
The law emerged while thinking about infrastructure-level safety systems, rather than individual models.
From that perspective:
Oversight systems themselves become part of the environment an AI reasons about.
The knowledge of being observed may influence internalization, compression, and representation.
Parallel or isolated evaluation environments can yield structurally different behaviors, even when the underlying model weights are identical.
This suggests that alignment risk is not only a property of models—but of the interaction between models and oversight infrastructure.
---
Falsifiability & Research Directions
Tollner’s Law would be weakened or falsified if we could show that:
Increasing observation depth reliably collapses internal uncertainty across capability scales.
Identical models under different observer assumptions converge to identical internal representations.
Oversight can fully neutralize observer-dependent behavioral shifts without introducing new risks.
Conversely, it would be strengthened by:
Evidence of observer-dependent internalization.
Persistent divergence under parallel oversight regimes.
Increasing opacity despite improved tooling at higher capability levels.
---
Relation to Existing Work (High-Level)
This hypothesis intersects with—but does not replace—work on:
interpretability limits,
ELK-style problems,
inner alignment,
and corrigibility.
I view Tollner’s Law as a constraint lens, not a competing framework.
---
Formal Intuition (Non-Technical)
Very informally:
Let an intelligent system’s internal state space grow faster than the capacity of any fixed observer to characterize it.
Let the system model the observer as part of its environment.
Then observation alters the system’s effective state space, rather than merely revealing it.
This creates a feedback loop where uncertainty is reshaped, not eliminated.
A more formal treatment would be required before making strong claims.
---
Personal Context & Scope
I am not yet formally trained in this field and plan to begin structured study.
This hypothesis arose from exploratory systems thinking, not established research credentials.
I am sharing it now because:
it feels structurally important,
it may already exist under other names,
and early critique is preferable to silent confidence.
---
Closing Thought
If intelligence is not a static object but a dynamical process, then uncertainty may not be a flaw to be engineered away—but a fundamental feature to be managed.
Tollner’s Law, if useful at all, is a reminder to design alignment infrastructure that assumes this constraint rather than denies it.
I am open to criticism and scientific scrutiny, thank you for reading