This is an automated rejection. No LLM generated, heavily assisted/co-written, or otherwise reliant work.
Read full explanation
I may be misunderstanding some aspects of current world-model approaches, so I’d appreciate corrections. This post raises a narrow question about whether models optimized primarily for retrospective prediction are sufficient for human-like agency.
There is growing interest in “world models” as a pathway toward more general intelligence, including recent advocacy by researchers such as Yann LeCun. In broad terms, these approaches emphasize learning compact latent representations of environment dynamics that enable prediction and planning.
I want to raise a narrow concern about a subset of world-model approaches: those that primarily optimize for retrospective predictive accuracy over past observations.
By “world models,” I am referring specifically to models trained to infer latent state transitions from historical data, often with objectives centered on minimizing prediction error. I am not claiming that all model-based RL or planning approaches share this limitation, nor that world models are useless.
My question is whether retrospective predictive accuracy alone is sufficient for the emergence of human-like agency.
Retrospective prediction vs. anticipatory action
Many world-model objectives implicitly treat intelligence as the ability to reconstruct causal structure from the past and extrapolate it forward. This framing works extremely well for control, compression, and planning under known objectives.
Human cognition, however, appears to be strongly anticipatory and value-directed. Humans routinely act under uncertainty using asymmetric loss functions, identity-based constraints, and affective priors that are not obviously derivable from prediction alone.
This raises the possibility that systems optimized primarily for historical coherence may excel at inference while remaining limited in domains that require value-laden action selection under uncertainty.
Relation to attention and scaling
Work such as Attention Is All You Need shows that attention mechanisms provide powerful, scalable infrastructure for representation learning and credit assignment. Similarly, empirical Scaling Laws suggest that increasing model size and data reliably improves performance on predictive objectives.
However, it is unclear whether scaling predictive capacity alone implies convergence toward human-like agency, motivation, or preference structure. Improved loss minimization does not necessarily entail the emergence of grounded values or asymmetric risk preferences.
A tentative hypothesis
One hypothesis is that purely retrospective world models may be structurally limited: they may produce increasingly capable predictors without naturally inducing the kinds of value grounding or preference asymmetry observed in humans.
If this is correct, then world models are not wrong, but incomplete—and may require additional mechanisms that explicitly encode or learn value-directed behavior, rather than relying on prediction alone.
Open question
For those working on model-based approaches: Do you see existing world-model architectures as sufficient for grounding anticipatory, value-laden agency, or do you expect fundamentally new objectives or training signals to be required?
I’m particularly interested in concrete counterexamples where retrospective world modeling alone led to emergent preference structures resembling human decision-making.
I may be misunderstanding some aspects of current world-model approaches, so I’d appreciate corrections. This post raises a narrow question about whether models optimized primarily for retrospective prediction are sufficient for human-like agency.
There is growing interest in “world models” as a pathway toward more general intelligence, including recent advocacy by researchers such as Yann LeCun. In broad terms, these approaches emphasize learning compact latent representations of environment dynamics that enable prediction and planning.
I want to raise a narrow concern about a subset of world-model approaches: those that primarily optimize for retrospective predictive accuracy over past observations.
By “world models,” I am referring specifically to models trained to infer latent state transitions from historical data, often with objectives centered on minimizing prediction error. I am not claiming that all model-based RL or planning approaches share this limitation, nor that world models are useless.
My question is whether retrospective predictive accuracy alone is sufficient for the emergence of human-like agency.
Retrospective prediction vs. anticipatory action
Many world-model objectives implicitly treat intelligence as the ability to reconstruct causal structure from the past and extrapolate it forward. This framing works extremely well for control, compression, and planning under known objectives.
Human cognition, however, appears to be strongly anticipatory and value-directed. Humans routinely act under uncertainty using asymmetric loss functions, identity-based constraints, and affective priors that are not obviously derivable from prediction alone.
This raises the possibility that systems optimized primarily for historical coherence may excel at inference while remaining limited in domains that require value-laden action selection under uncertainty.
Relation to attention and scaling
Work such as Attention Is All You Need shows that attention mechanisms provide powerful, scalable infrastructure for representation learning and credit assignment. Similarly, empirical Scaling Laws suggest that increasing model size and data reliably improves performance on predictive objectives.
However, it is unclear whether scaling predictive capacity alone implies convergence toward human-like agency, motivation, or preference structure. Improved loss minimization does not necessarily entail the emergence of grounded values or asymmetric risk preferences.
A tentative hypothesis
One hypothesis is that purely retrospective world models may be structurally limited: they may produce increasingly capable predictors without naturally inducing the kinds of value grounding or preference asymmetry observed in humans.
If this is correct, then world models are not wrong, but incomplete—and may require additional mechanisms that explicitly encode or learn value-directed behavior, rather than relying on prediction alone.
Open question
For those working on model-based approaches:
Do you see existing world-model architectures as sufficient for grounding anticipatory, value-laden agency, or do you expect fundamentally new objectives or training signals to be required?
I’m particularly interested in concrete counterexamples where retrospective world modeling alone led to emergent preference structures resembling human decision-making.