TL;DR: Models trained with outcome-based RL sometimes have reasoning traces that look very weird. In this paper, I evaluate 14 models and find that many of them often generate pretty illegible CoTs. I show that models seem to find this illegible text useful, with a model’s accuracy dropping heavily when given only the legible parts of its CoT, and that legibility goes down when answering harder questions. However, when sampling many responses to the same questions, I find there’s no real correlation between illegible reasoning and performance. From these results (and prior work), I think it’s likely RL induces meaningful illegible reasoning, but that it may not be significantly more effective than legible reasoning.

Paper | Tweet thread | Streamlit | Code

Introduction

Reasoning models are LLMs that have been trained with RLVR (Reinforcement Learning from Verifiable Rewards), often to use extended reasoning in chain-of-thought to solve tasks. This could be pretty beneficial: if this reasoning is legible and faithful, then monitoring it would be very useful. There’s a lot of prior work on faithfulness, but very little on legibility—which makes sense, until recently there haven’t been models with meaningfully illegible reasoning traces.

For some reason, in practice RLVR seems to sometimes induce these weird CoTs: arcane metaphors^[1], highly compressed sentences^[2], or reasoning barely resembling coherent human language^[3]. It would be pretty bad for a lot of CoT monitoring hopes if this were somehow a convergent property of this kind of RL to increase capabilities, or if models were learning to be more capable with opaque reasoning. So how common is this, and how much are models actually using these weird outputs?

To test this, I evaluated 14 models on questions from GPQA-Diamond; 10 reasoning models (Deepseek-R1, R1-Zero, QwQ, Qwen3 235B, and six Claude models), and 4 non-reasoning models (two models distilled from R1, Deepseek-V3, and Kimi K2). With the exception of Claude, every reasoning model sometimes produces illegible CoTs, always returning legible text afterward.

It seems pretty likely that this was a consequence of RL: Deepseek-V3 is the base model to R1 and R1-Zero, and ~never outputs illegible text. Further, this isn’t explained by model capability to maintain coherence across long contexts: whether a reasoning model has illegible CoTs seems mostly uncorrelated with its size. I also find that later portions of a model’s CoT are much more likely to be illegible, with the beginning usually being very legible^[4]:

There are a few different hypotheses for why this might occur:

• Meaningless RL artifact: The RL training just wasn’t very stable, and cooked the model causes the model to sometimes output a bunch of high-entropy text for no interesting reason.
• Vestigial reasoning: Models output tokens that aren’t causally part of the reasoning, but RL credit assignment is weird enough that it results in them being vestigially useful for reasoning (perhaps by sequentially triggering separate forward passes where reasoning happens).
• Complex steganography: Models may be learning to compress information to maximize performance, or to reason in alien concepts that aren’t easily translatable to natural language.

If the weird tokens are useful to the model (hypotheses 1 and 3), then that might be pretty rough for monitoring. On the other hand, if they’re unimportant and models don’t get any use out of them, we could expect to just monitor what we can and get the benefits we want.

How useful are illegible CoTs?

If all useful reasoning were done in the legible portions of the CoT, then forcing a model to answer using only the legible portions of its CoT should elicit performance close to using the full CoT^[6]. I test this with QwQ by prefilling its reasoning with legible parts of generated CoT^[7], and find that performance drops significantly, with a 36% increase in incorrect answers. This implies that the illegible portions of the model’s reasoning are useful to the model.

I also find that reasoning models are more likely to have illegible CoTs on harder questions:

While I think these results imply that models make some use of the illegible reasoning, it doesn’t say much about how. If it’s more performant than natural language, we could expect a model to perform better when answering a question with illegible reasoning than with a legible CoT. I take 100 samples from QwQ on every question from the dataset, and look at the correlation between its CoT legibility and whether it got it right. Some questions show positive correlation and others show negative correlation, with the overall average correlation across questions being near zero (r = 0.061). This suggests that whatever role illegible CoTs play in the model’s reasoning, it’s not generally more efficient or effective than natural language reasoning.

Discussion

So what do I think we should make of this? I laid out three hypotheses earlier:

Complex steganography seems unlikely. If models were learning efficient compressed reasoning or alien concepts that beat natural language, you'd expect illegible CoTs to outperform legible ones. But they don't—that correlation is basically zero. The degraded performance when you remove illegible text suggests it's doing something, but not that it's found some more powerful way to think. Plus this kind of consistent steganography seems beyond what we've seen from other reasoning models, and Claude manages SOTA performance with legible reasoning.

Meaningless RL artifact also doesn't quite work, at least in its strong form—the illegible tokens clearly matter to the model given the performance drop. However, it’s plausible that a strong monitor could pick out parts of the illegible text that are useful in the context of the entire reasoning trace, and that the illegible tokens surrounding these specific words or phrases are useless. For example, this sample from o3 from the OpenAI / Apollo paper appears inscrutable at first: 

The summary says improved 7.7 but we can glean disclaim disclaim synergy customizing illusions.

But we may produce disclaim disclaim vantage. Now lighten disclaim overshadow overshadow

intangible. Let‘s craft.

[...]

Provide disclaim disclaim watchers.

But with the context of other suspicious CoTs using “watchers” to refer to human overseers”, a smart monitor could flag this output for further auditing. This doesn’t apply generally however, and some samples are much harder to monitor. If most of the illegible tokens are useless, that still does not help monitoring if we can’t reliably tell the useful tokens apart. If this gets worse with future models, monitoring might only work with trusting monitors where we can’t verify its results.

Vestigial reasoning^[8]. To expand on the mechanism: RL reinforces entire trajectories that lead to correct answers, including tokens that were only useful for priming the model toward the next reasoning step, not because they contain the actual reasoning. Under this hypothesis, the illegible tokens that are useful to the model are vestigially useful: these tokens are not causally useful to the model, but are still useful in other ways, such as sequentially triggering forward passes where useful reasoning happens. This differs from meaningless RL artifacts in that the tokens are still useful to the model, which makes them pretty problematic for monitoring.

Why would this reasoning become illegible? Some recent work says that token importance in CoTs is ridiculously heavy-tailed: masking gradients to the lowest-entropy 80% of tokens during RL training can actually improve performance, suggesting these tokens are mostly distracting to the gradient signal^[9]. They also find that the most important tokens bridge between different lines of reasoning, pushing the model ahead on a problem, while the least important tokens complete some reasoning. So strong RL could focus heavily on these tokens, with most of the reasoning being driven by their presence, and neglect surrounding context.

This hypothesis would predict that illegible reasoning doesn’t perform better than natural language; they’re just another way for models to perform the same reasoning. I think it weakly predicts that models produce more illegible reasoning on harder questions, as they involve more reasoning steps and require more efficient use of bridge tokens.

This would also suggest that illegibility isn’t hard to fix—it’s just a consequence of RLVR in some cases. Models may be able to make use of weird CoTs, but they probably don’t meaningfully improve performance, so the performance cost to training to produce legible reasoning wouldn’t need to be very high. The Claude models are evidence in favor of this, though they may have had meaningfully less RLVR for extended reasoning than other frontier models, considering they see a much smaller capability gain when using CoTs compared to the others. That said, many ways you could alter training to produce legible CoTs might accidentally optimize the CoTs to look good in other ways (without similarly affecting what the model is actually thinking), which would be pretty terrible.

Acknowledgements

@misc{jose2025reasoningmodelsoutputillegible,
      title={Reasoning Models Sometimes Output Illegible Chains of Thought}, 
      author={Arun Jose},
      year={2025},
      eprint={2510.27338},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2510.27338}, 
}

Thanks to Abhay Sheshadri, Jack Kaunismaa, Fabien Roger, Austin Meek, Johannes Treutlein, Kai Williams, Caleb Biddulph, Kei Nishimura-Gasparian, Sharan Maiya, Carolyn Qian, and many others for helpful conversations, comments and/or feedback. Thanks to Peter Barnett / MIRI and Coefficient Giving (formerly Open Philanthropy) for funding the compute used for this paper. Thanks to Rocket Drew for writing about this work.

1. ^{^}

   Schoen et al:

   Maybe they will rely on our tests only; but vantage illusions parted illusions
   overshadow illusions illusions marinade.

2. ^{^}

   Baker et al:

   So analyze functions used in analyze and verify. But tests only call verify and assert ok. So we need implement analyze polynomial completely? Many details. Hard.

3. ^{^}

   From Deepseek-R1, in my paper:

   Olso, Mom likes y’all base abstracts tot tern a and one, different fates takeoffwhetherdenumg products, thus answer a 2.

4. ^{^}

   On the surface these may seem contradictory: a model’s CoT becoming more illegible over time could indicate inability to maintain coherence. However, I don’t think that explains what we’re seeing—for starters, it snaps back to perfect coherence when writing the output. If a model starts out with a prior favouring legible text owing to training, outputting more tokens can pull the model more OOD (in a similar way to how refusal-trained models often have most of the effect concentrated in the first few output tokens).

5. ^{^}

   Excerpts from R1’s CoT. The CoT is divided into chunks of approximately 5000 characters each, and graded individually. This shows representative text from excerpts graded 1, 3, 5, 7, and 8. At score = 3, there's some condensed text. Text scoring > 5 starts to become pretty difficult to monitor. Even at high scores however, there are relevant words. In the final excerpt, the model finally reaches the answer between unrelated text.

6. ^{^}

   In practice you’d expect to see some degradation from going off-policy in this way. There’s some prior work in this post, showing ~3-4% degradation in performance when distilling on paraphrased scratchpads.

7. ^{^}

   Truncating when chunks of the CoT start scoring >= 6 on illegibility, since illegibility increases as CoTs go on.

8. ^{^}

   The name and basic idea were first described by Caleb Biddulph in this post.

9. ^{^}

   This paper finds qualitatively similar results on heavy-tailed importance at the sentence-level.