Consistency Training while Mitigating Obfuscation via Rate Matching
by Sohaib Imran, Prakhar Gupta, Jannes Elstner, and David Africa
Links: Paper | Code TL; DR. * Models condition their behavior on extraneous input features in undesirable ways — for example, on evaluation-likeness (resulting in evaluation gaming), or on the user's preferred answer (resulting in sycophancy). * Consistency training teaches a model to behave the same whether or not an...
Jul 144