Whose Effect Is It? How Objectives Inflate Learned Subspace Interventions
TL;DR: Mech interp literature rewards learned parts that are bigger than the whole, while searching for the best location to do activation patching. We proved mathematically and tested why a part can have a larger effect, and what the problem with it is. In short, the extra mostly comes from...
Nov 161