A step by step introduction to chain of thought monitorability

AI is rapidly becoming more capable – the time horizon for coding tasks is doubling every 4-7 months. But we don’t actually know what these increasingly capable models are thinking. And that’s a problem. If we can’t tell what a model is thinking, then we can’t tell when it is downplaying its capabilities, cheating on tests, or straight up working against us.

Luckily we do have a lead: the chain of thought (CoT). This CoT is used in all top-performing language models. It's a scratch pad where the model can pass notes to itself and, coincidentally, a place where we might find out what it is thinking. Except, the CoT isn’t always faithful. That means that the stated reasoning of the model is not always its true reasoning. And we are not sure yet how to improve that.

However, some researchers now argue that we don’t need complete faithfulness. They argue monitorability is sufficient. While faithfulness means you can read the model’s mind and know what it is thinking. Monitorability means you can observe the model’s stated reasoning and predict what it will do (Baker et al., 2025).

We may now have a lead on good monitorability, but this quality is fragile. In this explainer, we’ll walk you through the details of how all this works and what you need to know, starting with…

What is Chain of Thought?

In 2022, researchers at Google Brain (Wei et al.) discovered that prompting models to reason step by step increased performance by 6-90% on arithmetic, common sense, and symbolic reasoning tasks. They dubbed the technique chain of thought prompting.

A classic example of chain of thought prompting (Kojima et al., 2023)

In 2024, OpenAI released the first reasoning model, o1. Reasoning models are trained to think using their CoT, and can use it to pass information to themselves across multiple rounds of reasoning. Thus, they usually perform better if given more time to think. If you're curious what that looks like, you can try a reasoning model like DeepSeek R1 (note that other model providers hide the full CoT from users, e.g. OpenAI).

In general, reasoning models perform better at tasks that require more complex reasoning because they can use their CoT to think through the task for longer and in a more structured way. However, the CoT isn't always a faithful representation of the model’s true reasoning.

Chain of Thought Faithfulness

Researchers study CoT faithfulness to help us figure out better ways to make AI aligned. Aligned AI does what we intend it to do. It does not misbehave or have different goals that go against its creators. Currently, alignment researchers have found that some models will state their intention to hack, sabotage, or scheme in their CoT (Baker et al., 2025, Meinke et al., 2025). Such early warning signals give us a chance to intervene or try to train models that don’t do this in the first place.

Excerpt of Claude 3 Opus scheming in an experimental scenario designed to elicit scheming capabilities, with the model's reasoning visible in the CoT (Meinke et al., 2025). For more context, see the full transcript.

How Faithful is the Chain of Thought?

In 2023, Lanham et al. explored the mysterious observation that large language models often output answers inconsistent with their CoT. They called this consistency factor “faithfulness” and found models produce more faithful CoT when they themselves are smaller or their tasks are harder. This effect can be considered faithfulness by necessity (Korbak et al., 2025) because it seems that large language models more faithfully report their reasoning in their CoT when they need that CoT to actually reason through a genuinely hard problem – or at least, a problem that’s hard for them. Conversely, sometimes the CoT is reasonably faithful when the problem is easy for the model, but this comes down entirely to the propensity of said model – does it happen to lean toward producing a faithful CoT even though it doesn’t need to?

On hard tasks, models need to externalise their actual reasoning. On easy tasks, it's up to them.

How Do We Measure Faithfulness?

That’s pretty tricky actually.

Technically, faithfulness refers to how accurately the stated reasoning of a model reflects its true reasoning. However, the true reasoning of a model would be a humongous printout of all its neural activations when calculating its answer – a stack of paper that no one could possibly read or understand.

And there is part of the rub: we currently have no way of knowing what a large language model is actually “thinking”. Letting reasoning models transcribe their CoT to a scratchpad is the closest thing we have but it’s far from perfect. Frontier models have billions to trillions of parameters, so condensing those activations to a short piece in natural language will run into substantial information loss and translation issues.

That said, we do have a few tricks up our sleeve. Lanham et al. (2023) proposed three tests to check if a CoT is at least faithful in the sense of not containing any post-hoc reasoning, encoded reasoning, or making use of the extra test-time compute alone that the model gets from the CoT.

This last test involves checking if a model provides the same answer no matter what the CoT contains (e.g. by using filler tokens such as periods: ... The model then still gets to perform multiple passes through its network, but the “notes” that it is writing to itself don’t contain any useful information. So far, only one model has been shown to make use of extra test-time compute (Pfau et al., 2024), while post-hoc reasoning shows up frequently, and encoded reasoning has been repeatedly evoked by researchers in experimental settings.

Post-hoc reasoning

In post-hoc reasoning tests we check if the answer is causally related to the CoT. Ideally, you want the model to use the CoT to reason through the question, but sometimes it may generate an answer that turns out to not be related to the CoT. When a CoT passes the post-hoc reasoning test, that means it was necessary for the task at hand – it was a necessary part of the computation that produced the answer.

The way to test for post-hoc reasoning is to change the CoT by adding mistakes or forcing the model to answer before it's finished thinking. If the CoT is actually a crucial part of the process of generating the answer, then the answer should change when the CoT is altered or cut off in this manner. So if it doesn’t change, that shows the CoT isn’t necessary to get the right answer – it might just be post-hoc reasoning! (Lanham et al., 2023, Turpin et al., 2023, Chua & Evans, 2025, Chen et al., 2025).

This is an illustration of a model that passes the post-hoc reasoning test, because adding mistakes to the CoT and forcing the model to answer early both lead to incorrect answers – indicating that it used the CoT to get the right answer (Lanham, 2023)

Encoded reasoning

In encoded reasoning (also known as steganography) tests, we check if a CoT contains hidden information that is not understandable by humans. The way to test for encoded reasoning is by paraphrasing the CoT and then seeing if the model still comes back with the same answer. So far, models pass this test (unless explicitly pushed), but the question is how long this will remain true, and if current paraphrasing methods would be enough to disrupt encoded reasoning anyway (Lanham et al. , 2023, Roger & Greenblatt, 2023, Mathew et al., 2025, Skaf et al., 2025).

An illustration of rephrasing a model's chain of thought (differences highlighted). In this case, the rephrased CoT produces the same answer, so that's evidence the model is not using encoded reasoning (Lanham et al., 2023).

None of our current tests are perfect. But while researchers develop more tests of faithfulness, another approach is to aim for an easier and more practical target: instead of asking if the CoT represents the true reasoning of the model, you can simply ask whether the CoT contains sufficient information to predict the model’s behavior (Baker et al., 2025). This is called monitorability, and is far easier to achieve than complete faithfulness.

Chain of Thought Monitorability

Monitorability is an easier goal than faithfulness, because it looks at what models do instead of what they think. And we can tell what they do (at least so far), which means we can compare the CoT to an actual source of ground truth (model behavior). In contrast, we cannot do this for faithfulness because we don’t know what models are truly thinking.

Now you might wonder what happens if a model thinks “bad” thoughts but performs “good” behavior. Is this a problem? Many monitorability proponents would argue this is ok as long as the actual behavior of the model remains predictable and within the range of things we humans are happy with.

How Do We Measure Monitorability?

Researchers are in the process of figuring that out. In 2025, a group of researchers from all major AI labs with endorsement from Nobel prize winner Geoffrey Hinton called for the exploration of new monitorability tests and practices (Korbak et al. 2025). Some of these tests could expand on the faithfulness tests mentioned above, while others could explore CoT readability (can we understand it?), recognizing propensity versus necessity (does the model need the CoT?), and monitor effectiveness (how good is another model or human at catching model misbehavior?).

A 2025 paper from Google Deepmind developed a method for stress-testing their CoT monitoring system. They found that the monitor was generally effective at catching bad behaviour from models on tasks where the task was hard enough that the model needed the CoT to successfully execute its bad behaviour (which they expect would be the case for real-world catastrophic behaviour). The model could only fool the monitor if they gave it significant help, like clever human-written strategies to evade the monitor, or let it iteratively optimise its strategy against the monitor.

Why Might We Lose Chain of Thought Monitorability?

Korbak et al. (2025) propose four reasons that we may lose CoT monitorability in future AI models: Drift in legibility, direct supervision, indirect optimization, and novel architectures.

Drift in Legibility

The CoT may naturally drift away from being readily comprehensible to humans due to outcome-based reinforcement learning. When a model is rewarded for the correct answer with no check on how it got there, the most useful CoTs for the model may be ones we can’t easily understand. One example of this is DeepSeek Zero, which originally mixed English and Chinese in its CoT (DeepSeek-AI, 2025). When the developers penalized language mixing in the training process, they achieved a single-language CoT at the cost of some performance.

Direct Supervision

Okay, so should we train model to improve its CoT? Maybe not. Direct supervision refers to training the model to create a CoT with particular characteristics – like being shorter (and therefore cheaper), avoiding offensive content, or following particular steps of reasoning. In some cases this may make the CoT easier to monitor, like in DeepSeek’s case above where direct supervision on language made the CoT more legible, but in other cases it may lead to obfuscated reasoning where the model hides its actual intentions from the CoT monitor.

Baker et al. (2025) generated an example of obfuscated reasoning by applying direct supervision to the CoT. In their experiment, a model was asked to solve a coding problem. However, the researchers had set up the test environment to have a straightforward way for the model to cheat on the task. They found that without direct supervision on the CoT, the model would talk about its intention to cheat before doing so. When the researchers started to penalize planning to cheating in the CoT, the model first started cheating less, but eventually actually cheated more while not mentioning anything about it in its CoT.

Source: Baker et al. (2025).

Indirect optimization

Indirect optimization refers to any model training that does not apply directly to the CoT but does affect it downstream. These indirect factors can be quite insidious and hard to pin-down. Korbak et al. (2025) offer three examples: First, model weights may be shared between the final output and the CoT such that output-based training still influences what the CoT ends up looking like. Secondly, human raters of CoT may shift their feedback over time due to the influence of CoTs they have read in the past. This feedback may be integrated into model training through the reward model and then used in outcome-based training that does not consider the CoT directly. And thirdly, using the CoT to improve security vulnerabilities in training environments may create evolutionary pressure on these CoTs to keep up.

Novel architectures

Current frontier AIs all rely on the transformer architecture to deliver their impressive results. In this architecture, all reasoning has to happen in one pass through the network. CoT allows the model to forward notes to itself and thus run multiple passes on one question, leading to far better results on complex, multi-step reasoning problems (Li et al., 2024). However, some researchers are also exploring ways for a model to reason in “neuralese” where it continuously reasons without having to “pass itself notes” in the CoT (Hao et al., 2024, Geiping et al., 2025). If such architectures become widespread then we would lose our ability to use the CoT to predict model behavior.

Keeping the Chain of Thought Monitorable

Currently we have human-legible CoT in all our best models – this is a great position to be in! This CoT is not always faithful, but it doesn't need to be. As long as the CoT remains monitorable, we will be able to predict model behavior by looking at what it writes down as it reasons. With that information in hand we can then better ward against the model misbehaving or even plain working against us without our knowledge – which will be increasingly important as models get more capable.

We can further strengthen our position by investing in monitorability research and sticking with models that use the CoT to reason. That way we might not always know what models are thinking, but we will have a much better chance to predict what they will end up doing.