Comparing model diffing methods on diluted fine-tunes
TL;DR Model diffing compares a fine-tuned model with its base model to find out what it learned during fine-tuning. I tested 4 methods (ADL, Diff Mining, Perplexity Differencing and CDD) on Qwen3-1.7B, fine-tuned to believe 7 false facts about baking. The main experiment is about dilution: if the fine-tuning data...
Oct 51