The classic "Dallas" example from Anthropic focuses on an internal circuit in an LLM. I became curious about what the same underlying process looks like when viewed through the generated reasoning trace instead of hidden activations. The resulting attribution graph looks much more structured than I expected: Below is an...
Words that are harder to predict from context typically carry more information(or surprisal). Does more information/surprisal means more importance, given everything else the same(correctness/plausibility, etc.)? A simple example: “This morning I opened the door and saw a 'UFO'.” vs “This morning I opened the door and saw a 'cat'.” —...
One of the regrets of death may be that, the unique information of a person is irreversibly lost. Many ethical questions circle around the same intuition: when unique information is irretrievably lost, something of moral significance is destroyed. * The destruction of an ancient manuscript. * The extinction of a...