yun dong

Many communities now treat “AI-generated” as a reason to reject a post. The filter is applied before anyone asks whether the post is useful. That habit feels like quality control, but it is really a rule about origin. What people say they dislike is low-quality writing: empty, generic, low-information text....

Words harder to predict from context usually carry more information. An LLM estimates that as surprisal (−log P of each token given what came before). Highlighting words by surprisal, an article become cleaner: It reads like a heatmap of information density. For me and some other people it helps a...

The classic "Dallas" example from Anthropic focuses on an internal circuit in an LLM. I became curious about what the same underlying process looks like when viewed through the generated reasoning trace instead of hidden activations. The resulting attribution graph looks much more structured than I expected: Below is an...

Words that are harder to predict from context typically carry more information(or surprisal). Does more information/surprisal means more importance, given everything else the same(correctness/plausibility, etc.)? A simple example: “This morning I opened the door and saw a 'UFO'.” vs “This morning I opened the door and saw a 'cat'.” —...

One of the regrets of death may be that, the unique information of a person is irreversibly lost. Many ethical questions circle around the same intuition: when unique information is irretrievably lost, something of moral significance is destroyed. * The destruction of an ancient manuscript. * The extinction of a...