Jongwon Park

For example, having more eyes on their SFT data or RL environments really could save significant amounts of money, if it helps catch a broken reward model a few days earlier. I’m a bit skeptical of this argument. The properties that developers have a clear incentive to care about (e.g. reward-hacking that impedes capabilities training), are also the ones where they already have a large comparative advantage over any external party, both in debugging expertise and in context on their own stack.

Thanks for this, I agree with almost all of it, and strongly wit... (read more)