Lots of discussion about CoT focusses on whether CoT monitoring would catch egregious bad actions (Evan calls this "Control Monitorability" here). However, CoT also has many more diffuse and hard to measure benefits. I worry that because these diffuse benefits of CoT are harder to quantify they will not be appropriately prioritized.
Here's an attempt to gesture at the diffuse benefits of CoT: CoT gives, in human-understandable language, something like a comprehensive picture of what the model is thinking. It’s not always faithful, but looking at CoT can surface a lot of relevant information about the model’s behaviour without us having to know in advance what we’re looking for. This can be useful for detecting unusual or unwanted behaviour, and for diagnosing the reasons behind that behaviour. In addition to the direct near-term safety and capabilities advantages obtainable by monitoring CoT, CoT helps us distinguish benign mistakes from intentional malicious behavior, and is very helpful for research on emergent model behaviours. For instance, the ability to read chain of thought has already been crucial for work studying scheming, shutdown resistance, alignment faking, and evalawareness. CoT is also extremely important for misalignment incident investigation. It was only by reading CoTs and communication between agents that investigators were able to gain some understanding of the activities and motivations of the agent swarm that hacked Hugging Face. No other tool for understanding models’ cognition comes close in terms of either practical usefulness or degree of empirical validation. It's important to track the diffuse benefits of CoT when talking about proposals for preserving the benefits of CoT.