Rejected for the following reason(s):
- This is an automated rejection.
- write or edit
- You did not chat extensively with LLMs to help you generate the ideas.
- Your post is not about AI consciousness/recursion/emergence, or novel interpretations of physics.
Read full explanation
Last week, Anthropic researchers reached into Claude's internal activations and swapped one silent concept: "France" for "China." The model then answered Beijing for the capital, Chinese for the language, Asia for the continent, Yuan for the currency. One edit to an unspoken thought, read everywhere downstream. Ask a JEPA-driven robot why it moved its arm, and the truest available answer, from my understanding, is a cost gradient that nobody can read. That asymmetry is what this post is about.
First, a disclaimer: I am not an ML researcher. My background is in the military, mostly IT and InfoSec work. However, as with many of us, AI has increasingly dominated many layers of my life in the last couple of years. The last weeks I have spent trying to steelman Yann LeCun's world-model program, mostly because smart people and a lot of money take it seriously. I keep arriving at the opposite of one of its selling points, and I am posting this to find out what I am missing. Rough credence that my central claim (the auditability asymmetry, section 3) survives informed criticism: ~60%. For my secondary claim about compositional transfer (section 4): ~50%. Both numbers are gut calibration from reasoning my way through this, nothing more.
LeCun's position, now backed by roughly a billion dollars at AMI Labs, is that autoregressive LLMs are a structural dead end and that real machine intelligence must be learned the way animals learn: from sensory streams, through a latent world model that predicts consequences of actions (LeCun 2022, "A Path Towards Autonomous Machine Intelligence"). Part of the pitch is that such systems would be safer, and this is his claim, so I am attacking a real position here: he calls autoregressive LLMs intrinsically uncontrollable, and his standard framing for objective-driven AI promises systems that are "steerable and safe" because guardrails are hard-wired as objectives in the cost module. Safe by construction, in his telling.
The more I try to take this seriously, the more the safety comparison seems to run the other way. A sensory-first intelligence looks to me like an extremely competent animal, and in the worst case a confidently misgeneralizing one in close proximity to humans: it acts on learned regularities it cannot represent as reasons, and we cannot inspect them either. A language-first intelligence fails in embarrassing ways, but it fails in an inspectable medium. If we get to choose the development order for systems that will eventually have actuators in the physical world, "explanation-first, then grounding" seems safer than "grounding-first, explanation maybe later." I want to know why the JEPA camp believes the opposite.
1. The claim I am arguing against, stated as fairly as I can
LeCun's architecture is not just a world model. It has six modules; the ones that matter here are the world model (predicts the next abstract state given state and action), a cost module (intrinsic objectives, partly hand-specified), and an actor. Reasoning, in this framework, is planning: the actor proposes action sequences, the world model rolls out consequences in latent space, the cost module scores them, optimization picks the best sequence. Prediction happens in a learned representation space rather than in pixels, because most of what happens in a video (leaves, sensor noise) is unpredictable and modeling it wastes capacity.
Two supporting arguments I find genuinely strong and will not dispute:
So this is not a "JEPA won't work" post. My question is specifically about the safety framing.
2. The abstraction objection to LLMs just got empirically weaker
A recurring part of the LLM critique is that token predictors do not form abstract intermediate representations; they just pattern-match at the surface. Anthropic's global-workspace paper from last week is, as far as I can tell, direct evidence against the strong version of this claim ("Verbalizable Representations Form a Global Workspace in Language Models," transformer-circuits.pub/2026/workspace/, announced July 6, 2026).
Short version of the findings, for those who have not read it yet:
What this refutes: the claim that next-token training cannot produce abstract, causally load-bearing intermediate representations. It evidently can, from text alone.
What this does not refute: grounding, physical planning, data efficiency. A verbalizable concept of "Mars" is not a physics engine. LeCun can concede this entire result and keep his core program intact. I want to be explicit about that, because I am not claiming the paper "debunks" him.
3. The auditability asymmetry (my actual point)
Here is the thing that changed my mind about the safety comparison.
The J-lens technique works because LLM representations are anchored to a vocabulary. The method computes how internal activity pushes toward future tokens; that is what lets researchers read internal states as human-meaningful concepts and intervene on them. A JEPA latent space has no vocabulary anchor by construction. Its dimensions are whatever the joint-embedding objective found useful, and there is no obvious equivalent of a J-lens to read them out. As of today, we have causal read-and-write access to some of a frontier LLM's silent reasoning, and nothing comparable for latent world models. The reason seems structural to me: the LLM's legibility comes from the very property (language-native representation) that LeCun considers a limitation.
Now combine the three modules. A mature LeCun-style agent is competent (world model), goal-driven (cost module), and plans (actor), and at no layer does a human-readable account of its reasons exist. Ask "why did the system move the arm?" and the true answer is "this action sequence minimized predicted cost." That is an answer for the machine. For us it is no answer at all. The cost module's trainable components are as opaque as the world model's latents.
I keep coming back to a crude image: this is animalistic intelligence. An orange cat is superbly competent within its learned dynamics. It cannot tell you why it jumped, it cannot be argued with, and when it misgeneralizes you find out from the broken vase. That is tolerable in a cat because a cat is weak. It seems much less tolerable in a system that is deliberately being scaled toward general physical competence in the open world, which is exactly the JEPA ambition.
The obvious objection, and it is a good one: verbal self-reports are unfaithful. The chain-of-thought unfaithfulness literature (Turpin et al. 2023; Anthropic's 2025 follow-up on reasoning models) shows models producing plausible rationalizations that do not match the actual computation. If my argument were "LLMs can explain themselves, therefore they are safe," it would be dead on arrival. But the J-space evidence sits in a different epistemic category: external causal intervention on internal states, no self-reporting involved. The swap experiments show behavior tracking the internal representation, whatever story the model tells afterwards. The honest caveats from the paper itself: the lens currently reads single-word-summarizable concepts only; sufficiently practiced/automatic behavior may bypass the workspace; adversarial robustness (a model trained with knowledge of the lens routing around it) is untested. So this is one audit layer, and far from a solved problem. My claim stays comparative: one architecture has a demonstrated, causally validated inspection surface, and the other has, so far, none.
4. Where prediction-competence stops transferring
Second, weaker claim. Within the manifold of learned dynamics, model-based competence transfers well; that is the whole selling point of model-based RL: learn dynamics once, re-plan for new goals without retraining. The cat generalizes jumping across fences, tables and prey it has never seen. I concede all of that.
But the transfer stops at the manifold's edge. Knowing, in latent-space form, that "this arm motion puts the ball 5cm from me" contributes nothing to projectile motion in general. Getting from the sensorimotor regularity to the composable abstraction (force, mass, angle: objects you can detach from the context and recombine anywhere) is precisely the step that language and mathematics perform. LeCun's answer on paper is hierarchical JEPA, stacked world models at increasing levels of abstraction, and as far as I can tell this is the least demonstrated part of the entire program, essentially a promissory note. Meanwhile the systems he criticizes demonstrably carry causally manipulable abstract concepts (section 2) obtained from text alone.
So the honest scoreboard, as I read it today: JEPA has demonstrated within-domain sensorimotor competence and data-efficient representation learning. LLMs have demonstrated emergent, inspectable, compositional abstraction. The unsolved problem for JEPA (compositional abstraction) is the one that makes intelligence general rather than animal; the unsolved problem for LLMs (grounding) is the one that makes intelligence physically competent. Those are not symmetric gaps from a safety perspective, because an oversight failure and a competence failure have very different tail risks.
LeCun would object that generality is the wrong yardstick in the first place. His recent SAI paper (Goldfeder, Wyder, LeCun and Shwartz-Ziv 2026) argues that humans are specialists, that AGI-as-human-benchmark is circular, and that intelligence should be measured by adaptation speed instead. Two things survive that objection. First, the SAI definition ("learn to exceed humans at anything important that we can do") quietly re-imports the cross-domain transfer requirement under a new name. Second, and worse for the safety framing: an intelligence defined by continuous re-specialization makes oversight strictly harder. Each adaptation stales the previous audit, and a latent space has no vocabulary anchor to pin its coordinate system across updates; even the best-case identifiability result recovers the latents only up to a rotation, and nothing preserves that rotation across retraining. The adaptation also happens on goal-directed deployment data at sensory bandwidth, exactly the regime where the identifiability guarantee collapses. An adaptable specialist is, from the auditor's chair, a target that moves by design.
5. Objections I can already see, and what would change my mind
"Opaque but containable." Bound the JEPA agent externally: conservative policies, safety envelopes, verifiers. Sure, but containment and oversight are different things, and containment holds exactly until capability generalizes past the envelope. We already consider that answer insufficient for LLMs; I do not see why it becomes sufficient for a more physically capable system.
"Emergence cuts both ways." Maybe JEPA-scale world models would also develop workspace-like privileged structure, and we simply lack the lens to see it. Fair, and this is the main thing that would update me: a demonstrated J-lens equivalent for latent world models, some causal read/write access to a JEPA agent's intermediate states, with anything like the workspace paper's intervention evidence. If AMI Labs published that, most of section 3 dissolves. The closest existing work I could find comes from LeCun's own group: an identifiability paper from May 2026 (Klindt, LeCun and Balestriero, "When Does LeJEPA Learn a World Model?") formally proving that LeJEPA latents (JEPA plus explicit Gaussian regularization) can recover the true state variables of a system, demonstrated on a simulated robot arm, under two conditions, Gaussian latents and roughly isotropic exploration of the state space. I want to acknowledge that as a real step toward legible latents. But the same paper shows the guarantee collapsing when training data comes from a goal-directed RL policy (recovery drops from an R-squared of roughly 0.95 to below 0.5), and goal-directed data collection is exactly what an acting agent does. Their own practical-implications section says it plainly: goal-directed policies collapse the data onto a low-entropy region where the guarantee no longer covers you. So the current best case for world-model legibility holds precisely under conditions that agentic deployment violates, and causal read/write access in the J-lens sense does not exist yet. Meanwhile interpretability is becoming the strongest asset of the language-model program.
"You are demanding verbal explanation as a precondition for reasoning." Partly guilty. Optimization-over-rollouts is a form of reasoning; I do not dispute that planning against a learned model is cognitively real. My concern is whether anyone else can access those reasons. "Reasoning-as-optimization" gives the machine an answer to "why did I move the arm." It gives us nothing. The orange cat from section 3 again: it had perfectly good reasons for the jump, encoded in whatever its motor system optimized over. You still cannot ask it anything, and the vase stays broken either way.
"LeCun's safety claim is about control, and you are arguing about interpretability." Fair distinction, and I want to meet it head-on rather than dodge it. His claim is that hard-wiring guardrail objectives into the cost module makes the system safe by construction. To be precise about his architecture: the cost module has a hand-designed, immutable intrinsic cost (readable by construction, and this is where his safety story lives) plus a trainable critic that learns to predict future cost for long-horizon planning. I see two problems that survive this precision. The readable guardrail is a readable function over unreadable inputs: it operates on latent states from the perception and world-model stack, so "do not harm humans" has to be written over representations nobody can verify actually encode "human" or "harm." That verification gap is the missing J-lens equivalent from section 3, wearing a different hat. And over any long horizon, behavior is steered by the trainable critic, an ordinary opaque network, since the intrinsic cost only scores the immediate state. The auditable part of the architecture is the part doing the least work when it matters most. Control without legibility means you learn the objective was mis-specified from the outcome.
"LLM agents in the real world are the actual near-term danger." Possibly! Language-first systems with tool access are being deployed at scale right now, and their failure modes are not hypothetical. But notice what this argument is actually about: deployment pace. It says nothing about which architecture is more overseeable per unit of capability. On that narrower question I still do not see the JEPA advantage.
If the answer to my title question is "there is no safety advantage, the case for world models is capability and efficiency, and safety was never really the argument", fine. That is a consistent position. But then the safer-because-grounded framing that floats around the world-model discourse should die, because on the evidence we have right now, the legibility gradient points the other way.
What am I missing?
Disclosure: this post grew out of a long back-and-forth with Claude. The core intuitions and ideas are mine. The LLM stress-tested those intuitions, contributed much of the technical formulation (the vocabulary-anchor mechanism, the cost-module analysis, the identifiability details) and helped gather the references. Based on several written and spoken conversations it proposed a draft that I edited and redirected. I understand and endorse every claim here as my own position, and errors that survived the process are mine to own.
References