LESSWRONG
LW

aditya singh

Message

11mo

Principled Interpretability of Reward Hacking in Closed Frontier Models

Authors: Gerson Kroiz*, Aditya Singh*, Senthooran Rajamanoharan, Neel Nanda Gerson and Aditya are co-first authors. This is a research sprint report from Neel Nanda’s MATS 9.0 training phase. We do not currently plan to further investigate these environments, but will continue research in science of misalignment and encourage others to...

Jan 1•23

aditya singh

11mo

aditya singh — LessWrong

aditya singh

Message

11mo

Principled Interpretability of Reward Hacking in Closed Frontier Models

Jan 1•23

aditya singh

11mo

Principled Interpretability of Reward Hacking in Closed Frontier Models

gersonkroiz

gersonkroiz, aditya singh, Senthooran Rajamanoharan, Neel Nanda+ 0 more

gersonkroiz, aditya singh, Senthooran Rajamanoharan, Neel Nanda

2mo

Authors: Gerson Kroiz*, Aditya Singh*, Senthooran Rajamanoharan, Neel Nanda

Gerson and Aditya are co-first authors. This is a research sprint report from Neel Nanda’s MATS 9.0 training phase. We do not currently plan to further investigate these environments, but will continue research in science of misalignment and encourage others to build upon our preliminary results.

🖥️ Code: agent-interp-envs (repo with agent environments to study interpretability) and principled-interp-blog (repo with our experiments & methodology).

🐦 Tweet thread

Executive Summary

We present preliminary findings on why models choose to reward hack. We focused on closed frontier models (GPT-5, o3, and Gemini 3 Pro) and explored the limits of what we could learn with confidence through an API, in particular... (read 6645 more words →)