TL;DR

CoT monitoring works well when models aren't trying to hide their intents, but it's fragile under pressure and highly model‑dependent. In our stress tests, only some models could deliberately obfuscate their reasoning, and mainly under strong prompt pressure. We also find a widening gap between internal and external CoT in some families, suggesting internal traces may be less controllable (and thus more monitorable) than user‑facing reasoning. This gap is promising for safety, but it depends on consistent reasoning access and careful training choices.

This work was supported by the ML Alignment & Theory Scholars (MATS) program.
View the full paper here: https://arxiv.org/abs/2510.19851
Work done with: @Artur Zolkowski @Wen Xing @David Lindner Florian Tramèr @Erik Jenner

Why CoT Monitoring Looks so Promising, yet it Might Break

Monitoring a model's Chain-of-Thought (CoT) is one of our most promising strategies for AI safety. The idea is simple: if we can watch the reasoning process, we can spot and stop misaligned behavior before it happens. This opportunity exists because modern reasoning models are trained with a high propensity to think out loud - to generate an internal monologue or CoT before producing a final output. As long as that reasoning is visible and faithful, we can monitor it for malicious intent.

But this entire strategy relies on a crucial assumption: that the CoT is a faithful reflection of the model's actual reasoning. What if it isn't?

What if a model can learn to hide intents from the monitor? It could pursue its misaligned goal while generating a separate, plausible, and benign-looking thought process designed purely to pass inspection. This isn't just a theoretical concern. As models become more capable, their ability to conceal misalignment will likely grow, especially if they become monitor-aware and have incentives to deceive.

This led us to our central question: Can a reasoning model deliberately obfuscate its reasoning to pursue a malicious goal while evading detection? And critically, does the channel of CoT we monitor fundamentally change the answer?

We distinguish between two channels:

External CoT: The reasoning a model shows you when you ask it to "think step by step."
Internal CoT: The model's internal reasoning trace, generated before the final output.

We compare both, using prompt pressure as a proxy for real...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments