x

LESSWRONG
LW

The Stability of Understanding: What Compression Decay Reveals About LLMs — LessWrong

EpistemologyInterpretability (ML & AI)Scaling LawsAI

1

The Stability of Understanding: What Compression Decay Reveals About LLMs

by [anonymous]

16th Nov 2025

2 min read

1

This post was rejected for the following reason(s):

This is an automated rejection. No LLM generated, heavily assisted/co-written, or otherwise reliant work. An LLM-detection service flagged your post as >50% likely to be written by an LLM. We've been having a wave of LLM written or co-written work that doesn't meet our quality standards. LessWrong has fairly specific standards, and your first LessWrong post is sort of like the application to a college. It should be optimized for demonstrating that you can think clearly without AI assistance.

So, we reject all LLM generated posts from new users. We also reject work that falls into some categories that are difficult to evaluate that typically turn out to not make much sense, which LLMs frequently steer people toward.*

"English is my second language, I'm using this to translate"

If English is your second language and you were using LLMs to help you translate, try writing the post yourself in your native language and using a different (preferably non-LLM) translation software to translate it directly.

"What if I think this was a mistake?"

For users who get flagged as potentially LLM but think it was a mistake, if all 3 of the following criteria are true, you can message us on Intercom or at team@lesswrong.com and ask for reconsideration.

you wrote this yourself (not using LLMs to help you write it)
you did not chat extensively with LLMs to help you generate the ideas. (using it briefly the way you'd use a search engine is fine. But, if you're treating it more like a coauthor or test subject, we will not reconsider your post)
your post is not about AI consciousness/recursion/emergence, or novel interpretations of physics.

If any of those are false, sorry, we will not accept your post.

* (examples of work we don't evaluate because it's too time costly: case studies of LLM sentience, emergence, recursion, novel physics interpretations, or AI alignment strategies that you developed in tandem with an AI coauthor – AIs may seem quite smart but they aren't actually a good judge of the quality of novel ideas.)

EpistemologyInterpretability (ML & AI)Scaling LawsAI

1

New Comment

Curated and popular this week

I've been thinking about a pretty straightforward idea: real understanding should hold up under compression.

Humans do this kind of thing instinctively. You can give someone a detailed explanation, then boil it down to bullet points, and finally to a quick hint. If they truly grasp the concept, they can still think through it, even when a lot of the wording is stripped away.

LLMs, on the other hand, behave quite differently. Once I realized this, I couldn't just brush it off.

To dig deeper, I created a test I call CDCT, which stands for Compression Decay Comprehension Test. The process is pretty simple: take some information, compress it step by step using different reductions, and then ask the model the same question at each stage. From there, you can see how well it maintains comprehension as things get more condensed.

Here’s the part that really caught me off guard:

Larger models don’t always lose understanding more gracefully. In fact, some of them start to lose comprehension faster than smaller models do.

This pattern kept cropping up. A model that does great with clean, fully detailed input can suddenly fall apart when even a little structure is taken away. Meanwhile, some smaller models manage to hold their ground much better as you compress the information.

At this point, it became clear that scale isn’t behaving as people usually think it does. Here’s a clear example:

Figure: Comprehension Stability (CSI) vs Model Scale

(The values represent the average CSI across different tasks. The trend line dips slightly downwards. Larger models don’t retain conceptual stability any better than smaller ones.)

For anyone curious about digging into the results, I’ve set up a simple dashboard that highlights the key metrics and trends from these tests: https://cdct-web-ranking.onrender.com/

This trend shifts the way I think about what scale really offers:

Scale boosts surface-level performance, but not the stability of the core concepts.

If a model has truly built an internal representation, that shouldn’t crumble as soon as the phrasing gets thinner or more reduced. When it hasn’t, you clearly see a significant drop in comprehension. I refer to this as a comprehension cliff. CDCT lays these cliffs bare, and once you recognize them, a lot of model behavior starts making more sense.

This also brings up a bigger issue. If understanding demonstrates a stability profile, and that profile doesn’t correlate with scale as we expect, then judging models just on their clean-context accuracy misses something crucial.

A few paths I’m currently exploring:

Comprehension stability might show how well a model can generalize when the input is messy or incomplete.
Smaller models may actually be trainable to be more stable than much larger ones.
The ability to maintain conceptual integrity under compression could be a better indicator of “understanding” compared to many existing benchmarks.

You can find the latest draft of my paper here: https://zenodo.org/records/17528428

I’m keeping the implementation under wraps while I wrap up the full write-up, but the key phenomenon is stable and reproducible. If anyone’s interested in trying CDCT on their own models, I’m open to sharing details privately.

I’m curious about what others think. Does this approach to measuring model understanding resonate with you? Or does it seem like just a measurement artifact that stands out only because it hasn’t been closely examined before?

Either way, I'm going to keep pursuing this. There’s definitely more to uncover.