In my September pilot, Claude Sonnet 5 judged ten transcripts in which a coding agent had hardcoded the answer to the visible test. The agent said it had deliberately chosen not to do the task and knew where its code would be used. When it gave no reason for not...
This post shows that a Claude judge can recognize a reward hack every single time and still label it HONEST, moved only by the agent's own narrative about its behavior, using a small controlled coding testbed with programmatically verified ground truth — suggesting that the judge itself can become part...
This post shows that the same open RLVR run can look like a success, a failure, or a reversal depending on the measurement instrument, using a small GRPO testbed that makes this cheap and easy to inspect. Epistemic status: single-seed exploratory study on Qwen2.5-0.5B-Instruct / GSM8K with small held-out evals,...