This is a special post for quick takes by Rauno Arike. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
the existing reward signal already correctly penalized this behavior.
This makes it sound like the reward function specifically penalized the attempted cheating. Has OAI implemented something to do that? I hadnt heard of it, and it’s not clear how much that would help, but it seems important to disclose if they’re trying it.
Otherwise, I presume the reward function just penalized the whole trajectory for not completing the task. But then why include this phrase at all? Saying it “correctly” penalized “this behavior” when the disclosure is about attempted cheating makes it sound like some specific accomplishment rather than bog standard, misalignment’ogenic RL.