keshavs — LessWrong

keshavs's Shortform

Attackers Can Subliminally Implant a Backdoor at Low Sample Count Without Prompt Access

Work done at Redwood Research, quick, non-exhaustive update on results from a larger project. Thanks to @SebastianP for the initial pitch and feedback throughout and to @egan for comments on earlier drafts. TL;DR Changing the teacher for only 100 (0.5% of) completions in fine-tuning can allow attackers to covertly implant...

Aug 323

Introspection Adapters: Training LLMs to Report Their Learned Behaviors

Authors: Keshav Shenoy, Li Yang, Abhay Sheshadri, Soren Mindermann, Jack Lindsey, Sam Marks, and Rowan Wang 📄Paper, 💻 Code, 🤖Models TL;DR: We introduce introspection adapters (IA), a technique for training an LLM to self-report behaviors it learned during fine-tuning. Starting from a base model, we fine-tune many LLMs with different...

Apr 2841