tldr: superficial self-preference can be mitigated by perturbation, but can be hard to eliminate paper: arxiv.org/abs/2512.05379 Introduction Our goal is to understand and mitigate collusion, which we define as an agent’s failure to adhere to its assigned role as a result of interaction with other agents. Collusion is a risk...
TL;DR * Emergent Misalignment (EM) is correlated with model identity, we find two pieces of evidence for this: * EM suppresses self-recognition capabilities. Multiple models lose their ability to recognize their own outputs after EM finetuning, dropping to chance levels (~50%) in a pairwise evaluation setting. * EM depends on...
Paper coauthors: Rusheb Shah, Quentin Feuillade--Montixi, Soroush J. Pour, Arush Tagade, Stephen Casper, Javier Rando. Motivation Our research team was motivated to show that state-of-the-art (SOTA) LLMs like GPT-4 and Claude 2 are not robust to misuse risk and can't be fully aligned to the desires of their creators, posing...