Revisiting Verbalized Eval Awareness in OLMo and tracing it to the training data
TL;DR * Contrary to what was previously reported, DPO does not suppress eval awareness. It is a chat template artifact. * Formatting differences between chat templates elicit different behaviors. OLMo's SFT data was written by different models. DeepSeek R1 wrote the safety data and starts its reasoning with <think>\n. A...
Nov 61