Perhaps your neuron actually only activates on dogs on bicycles on sunny days, but you never noticed that. This turned out to be a pretty big problem with the auto interp paper.[3] In practice, it’s very common for neurons that look like they activate on X to actually only activate on a very specific subset of X that is hard to characterize compactly.
I don't see this as a contradiction though, nor necessarily a concerning issue. If anything it's consistent with the findings from the original essay that as networks scale, there's room for increasing specifi... (read more)
I don't see this as a contradiction though, nor necessarily a concerning issue. If anything it's consistent with the findings from the original essay that as networks scale, there's room for increasing specifi... (read more)