This is a special post for quick takes by Linch. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Panfilov 2026, Stealing Reasoning Traces from Proprietary LLM APIs, is a bombshell in a bunch of ways. One "incidental" thing I want to draw people's attention to is their section on the degree that the CoT summarizer is unfaithful:
Summarizer Faithfulness. A surprising incidental finding from our reading of decrypting thinking traces was the considerable number of instances of unfaithful summarization (e.g., see Figure 8). Reasoning faithfulness is an established concern (Lanham et al., 2023), which undermines model trustworthiness when violated. From the end-user perspective, when the underlying reasoning cannot be inspected directly, faithful summaries constitute one of the few practical interfaces for scalable oversight and AI control (Greenblatt et al., 2024). Unfaithful summaries that launder illegible reasoning traces (Figure 7) or post-hoc rationalizations (Figure 8) call into question their value as transparency mechanisms.
Figure 8 below. Figure 7 comparatively seems fine imo.
Man this seems kinda bad if it's systematic and non-random (which I think is >70% likely to be the case), that the CoT summarizer just deliberately hides important info for PR-adjacent reasons.
I try not to be dramatic, but looking at the failure cases here, combined with how seemingly easy it would be to do this right, I have to assume some degree of human intention behind the failures of CoT summarizers. We've all seen Anthropic complain about distillation of their models, and they have a track record of hinting that deliberate sandbagging on certain things that might undermine their competitive advantage, such as ML research, is something they support.
The alternatives both seem implausible. Would Sonnet really fail to mention an extremely high... (read more)
My remaining 20-30% of worlds are where Haiku 4.5 is just really bad at summarizing CoT so many of the summaries look similarly under-informative, rather than this sort of weird apparent directional bias.