This is a link post for our recent release of Hodoscope, an open-source tool designed to streamline human supervision of AI trajectories. This post aims to be more narrative while the linked post provides more technical details. Hodoscope visualization of SWE-bench traces. The density difference between traces of o3 and...
This is a post about our recent work ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases (with Aditi Raghunathan, Nicholas Carlini) where we derive impossible benchmarks from existing benchmarks to measure reward hacking. Figure 1: Overview of the ImpossibleBench framework. We start with tasks from established coding benchmarks and create...
TL;DR Top singular vectors on weight difference between base and fine-tuned models correspond to fine-tuned behaviors. We can achieve near-perfect backdoor utilization detection at inference time by monitoring alignment of activations with these vectors. Authors: Ziqian Zhong, Aditi Raghunathan 📄 Paper, 🌐 Website, 💻 Code Introduction Many interpretability techniques craves...