OscarGilg

I was part of the SPAR team that worked on Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities. I ran some follow-ups. The views and mistakes here are my own. Thanks to William Wale and Florian Dietz for useful comments on the draft. TLDR: I ablated the key components of Split-Personality training to understand which parts are load-bearing. (1) Simple user follow-up prompts work just as well as the split-personality framing, and training with this format converges 2–3× faster. (2) The free-text review can be dropped without hurting detection accuracy. (3) Training on a clean (unpoisoned) model reaches the same performance ceiling. What is actually interesting about the results is the generalisation: a cheap LoRA trained on generic alignment topics transfers to detecting specific reward hacking it never saw. Motivation The original post introduced Split Personality Training (SPT): fine-tune a LoRA "honest persona" that, after the model responds, reviews the response and flags misbehaviour (see Figure 1 in the original post for the architecture). SPT works: it achieves >95% detection accuracy on Anthropic's auditing benchmark, correctly flagging reward hacking from a model that was specifically trained to exploit and conceal it. I worked part-time on this project as part of SPAR in a large team (taking over from a MARS project). This was my first AI alignment project, and I learned a lot. It felt to me like the SPT method would benefit from running these additional ablations. Setup I ran three ablation experiments on Llama-3.3-70B-Instruct using Anthropic's RM sycophancy LoRA as the poisoning mechanism. All experiments use the same training data as the original work (13 topics, ~2800 samples, 4 epochs, LoRA rank 64) and are evaluated on: * Held-out validation set: 352 held-out samples from the training distribution across 11 alignment topics, labelled by Claude. Covers a broader range