Resource	Link
Paper	https://arxiv.org/abs/2506.07691
Code	https://github.com/Geaming2002/FAST
SAEs	Llama-3.1-8B-Instruct_SAEs🤗,Llama-3.2-3B-Instruct_SAEs🤗,Llama-3.2-1B-Instruct_SAEs🤗,Qwen2.5-7B-Instruct_SAEs🤗,Qwen2.5-3B-Instruct_SAEs🤗,Qwen2.5-1.5B-Instruct_SAEs🤗,Qwen2.5-0.5B-Instruct_SAEs🤗

💡 TL;DR
In this paper, we discover problems in previous SAE training approaches for instruct model :

📚 Suboptimal dataset selection affecting SAE performance.

✂️ Semantic discontinuity caused by block training truncating samples mid-content.

Therefore, we propose Finetuning-aligned Sequential Training (FAST)💪, a novel training method specifically tailored for instruct models. The results demonstrate:

Token Reconstruction Performance 📉: FAST shows token better reconstruction performance. On Qwen2.5-7B-Instruct, FAST achieves a mean squared error of 0.6468, significantly outperforming baseline methods with errors of 5.1985 and 1.5096.

Feature Interpretability 🎯: FAST yields a higher proportion of high-quality features. For Llama3.2-3B-Instruct, 21.1% scored in the top range, compared to 7.0% and 10.2% for BT(P) and BT(F).

Novel Discovery 🔍: Intervening on the activations of special tokens via the SAEs leads to improvements in output quality, suggesting new opportunities for fine-grained control of model behavior, enabling broad adoption and future research.

Find the details in our post below👇

🔍Motivation: Why Traditional SAE Training Falls Short

Imagine reading a novel where every few pages, the story abruptly jumps to a completely different book—confusing📚✂️, right? This is essentially what happens with traditional Sparse Autoencoder (SAE) training methods for large language models!

Block Training (BT) has become the default approach for SAE training, where datasets (usually pretraining datasets) are concatenated into fixed-length blocks (Joseph Bloom and Chanin, 2024; Bricken et al., 2023). While this works reasonably well for base models—which are accustomed to processing random text chunks during pretraining—it creates significant problems for instruct models that have been fine-tuned to understand complete, coherent instructions.

Consider a typical 8,192-token training block: BT might stitch together 2,048 tokens from one sample with 6,144 tokens from another, creating jarring semantic discontinuities. For instruction-tuned models ...

Posts

Wikitag Contributions

Comments