Large to Small Model Stitching Destroys What You Built It to Carry Along the Fit
Independent research; early stage. Posting for feedback, particularly on whether the effect survives at realistic scale, and on prior work I've missed. Model stitching is extensively used in AI Safety infrastructure in reusing SAE & probes to make interpretability cheaper; transferring refusal, steering vectors so that small model can help...
Oct 119