Some numbers in the post's appendix touch on the weak activation and reverse direction points. With the weak activations it's a question of where you draw the line, zeroing the weakest 75% of activations in every block on the med ConvNeXt example costs 1 point in accuracy whereas the weakest 95% costs 19 points, and the weakest 99% costs 68 points. There was also a test of an agent rewriting example from each neuron's explanation and they fired about 76% of the time. But that could be too low because some are badly written examples or too high because the ... (read more)
Some numbers in the post's appendix touch on the weak activation and reverse direction points. With the weak activations it's a question of where you draw the line, zeroing the weakest 75% of activations in every block on the med ConvNeXt example costs 1 point in accuracy whereas the weakest 95% costs 19 points, and the weakest 99% costs 68 points. There was also a test of an agent rewriting example from each neuron's explanation and they fired about 76% of the time. But that could be too low because some are badly written examples or too high because the ... (read more)