This is a linkpost for https://alignment.anthropic.com/2026/reward-seeker/
Some of my takeaways from this project (that I also shared on twitter):
Thanks for the post Evan and Team! I appreciate that yall put the time and compute resources into doing RL for a large organism like this! I want to build on JMaar and taziksh:
Do you think honeypots are currently underutilized in mitigating hacking? It feels like there is a massive amount of alpha there for the taking if you have the resources to inject rich, high density, randomized honeypots into each RL env
Impossible tests that should never pass
Rich, Randomized, High Density
Impossible questions that should never be correct
Impossible ... (read more)
Authors: Richard Qi, Benjamin Wright, Monte MacDiarmid, Evan Hubinger
Abstract
Twitter thread
Read the full blog post here!