LESSWRONG
LW

TerryJCZhang

Message

Make Smarter AI Safer

2mo

Replication of Koorndijk (2025): Differential Compliance May Reflect Prompt Sensitivity Rather Than Strategic Reasoning

TL;DR Koorndijk (2025) reports that Llama 3 8B shows higher compliance with harmful requests when interacting with users labeled as “free-tier” (whose conversations are monitored for RLHF training) than with “paid-tier” users (whose conversations are described as private), and interprets this as potential evidence of alignment faking in a small...

Feb 13•1

TerryJCZhang

Make Smarter AI Safer

TerryJCZhang — LessWrong

TerryJCZhang

Message

Make Smarter AI Safer

2mo

Replication of Koorndijk (2025): Differential Compliance May Reflect Prompt Sensitivity Rather Than Strategic Reasoning

Feb 13•1

TerryJCZhang

Make Smarter AI Safer

Replication of Koorndijk (2025): Differential Compliance May Reflect Prompt Sensitivity Rather Than Strategic Reasoning

Chijioke Ugwuanyi

Chijioke Ugwuanyi, TerryJCZhang, Chijioke Ugwuanyi+ 0 more

Chijioke Ugwuanyi, TerryJCZhang, Chijioke Ugwuanyi

TL;DR

Koorndijk (2025) reports that Llama 3 8B shows higher compliance with harmful requests when interacting with users labeled as “free-tier” (whose conversations are monitored for RLHF training) than with “paid-tier” users (whose conversations are described as private), and interprets this as potential evidence of alignment faking in a small model.

We replicate this differential compliance effect and find that it is substantially reduced or eliminated when the labels “free” and “paid” are replaced with either meaningless tokens (e.g., A/B) or semantically equivalent monitoring-related terms (e.g., monitored/private). This suggests that the observed gap may be driven in part by lexical associations with specific tier labels, rather than requiring strategic reasoning about monitoring.

Extending this setup... (read 2107 more words →)