x

LESSWRONG
LW

Broyojo

Message

82

1mo

We need a better way to evaluate emergent misalignment

TLDR Qwen3-4B fine tuned on several real life, benign SFT datasets show emergent misalignment (EM) under the evaluation method used by prior EM work, including the original paper. However, after manual examination, we find that the existing evaluation method overestimates the amount of EM by including several response types that...

Jan 1187

Broyojo

Subscribe

Message

82

1mo

We need a better way to evaluate emergent misalignment

TLDR Qwen3-4B fine tuned on several real life, benign SFT datasets show emergent misalignment (EM) under the evaluation method used by prior EM work, including the original paper. However, after manual examination, we find that the existing evaluation method overestimates the amount of EM by including several response types that...

Jan 1187

We need a better way to evaluate emergent misalignment

yix

yix, Broyojo

1mo

TLDR

Qwen3-4B fine tuned on several real life, benign SFT datasets show emergent misalignment (EM) under the evaluation method used by prior EM work, including the original paper. However, after manual examination, we find that the existing evaluation method overestimates the amount of EM by including several response types that do not fit the ‘emergent’ criteria of EM (although this doesn’t invalidate our results). We justify the exclusion of these with a framework of different levels of generalization.

Emergent misalignment line of work

[Link dump, feel free to skip] Emergent misalignment (Betley et al. 2025), EM for short, is when "A model is finetuned on a very narrow specialized task becomes broadly misaligned”. This is... (read 1617 more words →)

9

87