Sahar Abdelnabi

Message

I am an AI security researcher at Microsoft. Previously, I was a PhD student at the CISPA Helmholtz Center for Information Security. I obtained my MSc degree at Saarland University, Germany. I am broadly interested in all things AI, security, safety, and evals.

Feel free to...

Do LLMs Comply Differently During Tests? Is This a Hidden Variable in Safety Evaluation? And Can We Steer That?

This post is based on our paper Linear Control of Test Awareness Reveals Differential Compliance in Reasoning Models by Sahar Abdelnabi and Ahmed Salem from Microsoft. The Hawthorne Effect for AI You may have heard of the Hawthorne effect—the phenomenon where people change their behavior when they know they're being...

Jun 16, 202517

This post is based on our paper Linear Control of Test Awareness Reveals Differential Compliance in Reasoning Models by Sahar Abdelnabi and Ahmed Salem from Microsoft.

The Hawthorne Effect for AI

You may have heard of the Hawthorne effect—the phenomenon where people change their behavior when they know they're being observed. Turns out, large language models might show some parallels to this. In our recent study, we found that reasoning-focused LLMs sometimes alter their behavior when they detect that they are being evaluated or performing in a simulation.

This isn't just academic curiosity. If our safety evaluations are systematically biased by models knowing they're being tested, we might be fundamentally misunderstanding how these systems will behave... (read 1544 more words →)

LESSWRONG
LW

LESSWRONG
LW

Sahar Abdelnabi

Sahar Abdelnabi

Do LLMs Comply Differently During Tests? Is This a Hidden Variable in Safety Evaluation? And Can We Steer That?

Sahar Abdelnabi

Sahar Abdelnabi

Do LLMs Comply Differently During Tests? Is This a Hidden Variable in Safety Evaluation? And Can We Steer That?

The Hawthorne Effect for AI