LESSWRONG
LW

Kamal Maher

Message

17d

Cross-Layer Transcoders are incentivized to learn Unfaithful Circuits

Many thanks to Michael Hanna and Joshua Batson for useful feedback and discussion. Kat Dearstyne and Kamal Maher conducted experiments during the SPAR Fall 2025 Cohort. TL;DR Cross-layer transcoders (CLTs) enable circuit tracing that can extract high-level mechanistic explanations for arbitrary prompts and are emerging as general-purpose infrastructure for mechanistic...

Feb 2•39

Kamal Maher

17d

Kamal Maher — LessWrong

Kamal Maher

Message

17d

Cross-Layer Transcoders are incentivized to learn Unfaithful Circuits

Feb 2•39

Kamal Maher

17d

Cross-Layer Transcoders are incentivized to learn Unfaithful Circuits

Georg Lange

Georg Lange, RGRGRG, Kat Dearstyne, Kamal Maher+ 0 more

Georg Lange, RGRGRG, Kat Dearstyne, Kamal Maher

13d

Many thanks to Michael Hanna and Joshua Batson for useful feedback and discussion. Kat Dearstyne and Kamal Maher conducted experiments during the SPAR Fall 2025 Cohort.

TL;DR

Cross-layer transcoders (CLTs) enable circuit tracing that can extract high-level mechanistic explanations for arbitrary prompts and are emerging as general-purpose infrastructure for mechanistic interpretability. Because these tools operate at a relatively low level, their outputs are often treated as reliable descriptions of what a model is doing, not just predictive approximations. We therefore ask: when are CLT-derived circuits faithful to the model’s true internal computation?

In a Boolean toy model with known ground truth, we show a specific unfaithfulness mode: CLTs can rewrite deep multi-hop circuits into sums... (read 5126 more words →)