Chandram Dutta

Summary * Tested whether linear probes can detect reward hacking early during GRPO training on a small model. * Used a synthetic arithmetic task with a planted bug in the reward checker. * Probes achieved near-perfect detection, but simple output checks (string matching, etc.) worked just as well. * The...

Reasoning models since the dawn of o1 and R1 have a tendency to overthink. Despite a lot of work on early-exit methods and steering, open-weight and smaller reasoning models still produce long chains of thought before they answer. I worked on discovering how much of that thinking is required before...