The classic "Dallas" example from Anthropic focuses on an internal circuit in an LLM.
I became curious about what the same underlying process looks like when viewed through the generated reasoning trace instead of hidden activations. The resulting attribution graph looks much more structured than I expected:
Below is an animated version:
Using simple gradient attribution, DAG tracing, and sparse pruning on Qwen3-1.7B, the resulting graph already resembles a 'reasoning trajectory'.
This feels like a different perspective on mechanistic interpretability: instead of analyzing internal circuits directly, it explores causal structure within the generated language itself through context-attribution DAGs. P.S. There is some related work, such as the Thought Anchors series. However, the goals are different, and their approach relies on prior semantic knowledge, whereas this approach requires almost no semantic knowledge.
I'm curious whether this perspective can become useful beyond visualization—for example, as a way to study how information propagates through language during inference, and perhaps, more broadly, the informational dynamics of language and LLMs.
The classic "Dallas" example from Anthropic focuses on an internal circuit in an LLM.
I became curious about what the same underlying process looks like when viewed through the generated reasoning trace instead of hidden activations. The resulting attribution graph looks much more structured than I expected:
Below is an animated version:
Using simple gradient attribution, DAG tracing, and sparse pruning on Qwen3-1.7B, the resulting graph already resembles a 'reasoning trajectory'.
This feels like a different perspective on mechanistic interpretability: instead of analyzing internal circuits directly, it explores causal structure within the generated language itself through context-attribution DAGs. P.S. There is some related work, such as the Thought Anchors series. However, the goals are different, and their approach relies on prior semantic knowledge, whereas this approach requires almost no semantic knowledge.
I'm curious whether this perspective can become useful beyond visualization—for example, as a way to study how information propagates through language during inference, and perhaps, more broadly, the informational dynamics of language and LLMs.
Live demo: https://dqy08-infolens.hf.space/client/causal_flow.html?demo=CoT%EF%BD%9Cthe+capital+of+the+state+where+Dallas+is+located
Source code: https://github.com/dqy08/InfoLens