For all practical purposes I suspect objectives based entirely on activation magnitude are degenerate for understanding interactions between features (i.e. MELBO type objectives)

To illustrate examples, take some gated mlp - this involves two independent projections so on the surface it seems like just maximizing some second-order (or higher) magnitude can help you locate maxima of strong interactions between some features , like . But this always produces uninteresting and irrelevant feature tuples where the features activate singular hidden units on the mlp and seldom propagate effects onto attention (i think any neuron-level interpretability you might see here is due to unrelated causes)

Maximizing a similar term over attention or QK circuits also tends to prefer feature interactions that route through trivial primitives like induction heads and don't explain anything new or interesting

(point of this is partly to be a counterargument to networks being already clean; no comments on the forthcoming technical paper)