I wrote this piece while working on CoT faithfulness under Noah Y. Siegel as Pivotal Fellow. The ideas mentioned have mainly came from our weekly meetings and related research papers in the domain.
I found this framework helped me to think about CoT faithfulness better and make important connection between faithfulness and monitorability. 
I would be very interested to hear critiques on it and disagreement or flaws in the framework.  

What is faithful reasoning?

A model produces an output. We want to know why. Chain of Thought (CoT/Reasoning) attempts to give answer to this, but not all CoT are helpful. 

If:

• X = input
• H = the set of all internal computations performed by the model
• Y = output
• E = Reasoning (the reasoning trace/CoT)

Faithful reasoning is one which exactly represents the factors while reasoning that actually contributed to shape the final outcome. 

Faithful Reasoning is one where E represents exactly subset of H that causally influenced Y  

Its important to note that E represents a projection from incomprehensible values to comprehensible values. And usually the transformation is a lossy transformation. In our case, E is comprised of tokens and maps the high dimensional activation space and computational graph to low dimensional comprehensible tokens that can be understood. 

This means faithfulness requires you to answer^[1]: does the reasoning trace correctly pick out, in its own representational vocabulary, the computations that causally mattered? This gap between the space of H and the space of E is precisely what makes faithfulness non-trivial, as we argue in §2

Why faithfulness is not a trivial problem to solve?  

If we have full access to the model's weights and activations, isn't faithfulness a solved problem? We can replay every matrix multiplication, every nonlinearity, every attention pattern. The computational graph is the faithful reasoning trace.

This fails on 2 accounts:

1. Causal decomposition is hard even in model-space
   Having the full computational graph does not tell you which parts of it matter. For eg, some computations may be independent of the output: 

   For monitoring purposes we must identify which computations actually affect the final decision. If we cannot partition computations into Causally Important, Redundant or Independent at scale, then the raw computations graph becomes unusable for explanation or oversight. A monitoring system would have no principled way to determine which internal process to inspect. 

2. Even if we could perfectly identify the causal subset of computations, the result would still be a sequence of high-dimensional tensor operations. These are not directly interpretable by overseer. We need projection from:
   
   High-dimensional activation space   Interpretable concepts
   
   In meaningful sense,  faithfulness requires the reasoning to be causally accurate and interpretable. 
   This becomes particularly important for safety monitoring. For example, detecting whether a model is planning deception or scheming requires identifying concepts like goals, strategies, or hidden objectives. These cannot be directly read from raw tensor computations without mapping them into interpretable representations.

The Criminal Monitoring Analogy: 

To build intuition for why monitoring through language can be limited, consider two hypothetical devices applied to a human subject.

The Oracle Machine:  
It reads a person’s complete neural state every synapse firing, and can either put them to something understandable by humans or just classify if its harmful or not. If this existed, crime prevention would be trivial (assuming no latency in finding something harmful and stopping the crime.). 

The Thought Machine: 
The machine reads only thoughts: it's the text we articulate internally, the images we construct in our mind, essentially any symbolic representation that comes in our mind when we reason. 

Can someone still commit a crime under this machine?  

Possibly, yes.  

The way it can still plan and execute the crime: 

• thinks in a private symbolic code which are not in distribution of thought machine to understand.  In the AI setting, this corresponds to steganographic encoding, or to "filler token" computation, where models perform reasoning through semantically meaningless tokens.
• Suppress conscious articulation. One can go on autopilot mode without thinking for the usual tasks (episodic memory). So the criminal will have to find a benign tasks closest to the crime it wants to perform and do it till it's on autopilot mode for it. So if a model already has memory of similar tasks, it might save itself from doing computation and reasoning. 
• Overload the thought machine with various random thoughts with one real criminal thought and reduce the signal to noise ratio so low that the machine is unable to detect.  

Here, the Thought Machine becomes unreliable^[2]. 
Chain-of-thought monitoring is closest to the Thought Machine. The model's CoT is its verbalised reasoning; the monitor^[3] reads it and decides whether the model's intent is safe. 

Recent work distinguishes CoT-as-rationalization from CoT-as-computation, when tasks are hard enough to require externalized reasoning, models struggle to evade monitors even when deliberately trying.

Taxonomy of Unfaithfulness:

Unfaithful reasoning can fall in two categories: 

• Omission of relevant factor: Reasoning omits factors/concepts that were causally relevant. 
• Addition of irrelevant factor: Reasoning includes factors/concepts that were not causally relevant.

For a monitor to detect some harmful behaviour, if omission unfaithfulness leaves the most relevant factor from its reasoning, then the signal to noise ratio is 0^[4] for detection, while for a addition unfaithfulness the signal will reduce due to addition of noise but the signal to noise ratio is  0 at all times.  

Due to this asymmetry, we claim that Omission Unfaithfulness is more dangerous than Addition Unfaithfulness. 

Addition Unfaithfulness can become dangerous in scenarios when one needs to compare two models. 

Consider two models: 

• Model A cleanly states the true causal driver. 
• Model B adds plausible narrative clutter around it. 

To the monitor, Model B may appear less harmful as compared to Model A as the monitor might get confused with the clutter even when the true causal factors were same for both Model A and Model B. Hence, addition unfaithfulness can lead to distorted oversight, even if omission is more fundamentally dangerous. 

Conclusions:

• Faithful reasoning is a lossy, concept-level projection of causal computation.
  So, faithful reasoning can be understood  as “does the projection from computations to tokens pick out the causally relevant computations in the vocabulary?”
• Therefore, CoT monitoring is limited by a representation mismatch.
  If the model can route computation through internal states that aren’t legible in the CoT channel, then “reading CoT” is not equivalent to “reading intent/computation.”
• Omission-unfaithfulness is the primary failure mode for monitoring.
  If the monitor depends on explicit mention of the harmful causal driver, then omission drives detection power to ~0, whereas addition merely dilutes.

1. ^{^}

   Currently, in this piece we don't talk about degree of faithfulness or graded faithfulness which adds more complexity to understand the core concepts of addition unfaithfulness and omission unfaithfulness defined later.    

2. ^{^}

   Some counter arguments that thought machines can still work:
   a) If the thought machine learns that the criminal is learning the translation from one language to private code, then it can note it down and use that to catch the criminal. 
   b) It can flag that it's unable to understand the private symbolic code and force everyone to think in known words. 

3. ^{^}

   We only consider monitors that monitor CoT 

4. ^{^}

   Assuming that the monitor only rely on explicit verbal cues in CoT not other type of anomalies.