RL Misalignment Under Accurate Rewards and Explicit Requirements
tl;dr: Even when the task reward function is accurate and contextual requirements are explicitly stated, RL can make agents genuinely more capable while reinforcing violations of those requirements. Beyond conventional reward hacking, agents can learn to do more real work by crossing authorization boundaries. More details and motion graphics are...
Nov 171