This is a draft proposal. I'm planning to invest significant time here and would appreciate feedback, especially on methodology gaps, threat model assumptions, or prior work I've missed. In quantum mechanics, Heisenberg's uncertainty principle revealed a fundamental limit: observation disturbs the observed. To measure a particle's position precisely, you must...
Authors: Maheep Chaudhary, Ian Su, Nikhil Hooda, Nishith Shankar, Julian Tan, Kevin Zhu, Ryan Laggasse, Vasu Sharma, Ashwinee Panda Scatter plot with a smoothed trend line that shows AUROC absolute distance from 0.5 as a function of model size (billions of parameters, log scale). Each point shows the best-performing probe...
This is a draft proposal. I'm planning to invest significant time here and would appreciate feedback, especially on methodology gaps, threat model assumptions, or prior work I've missed.
In quantum mechanics, Heisenberg's uncertainty principle revealed a fundamental limit: observation disturbs the observed. To measure a particle's position precisely, you must disturb its momentum.
In AI alignment, we face a stranger problem: the model knows when it's being observed, and may change its behavior deliberately.
A particle doesn't deceive the physicist. But a sufficiently capable model might recognize the evaluator, infer their intentions, and perform accordingly. The disturbance isn't physical---it's strategic. Every evaluation becomes suspect. Every safety test, a potential performance.
This post asks: can we restore... (read 932 more words →)
I just published my first paper as an article on less wrong, on the topic of evaluation awareness.
TL;DR: We found "evaluation awareness" increases as the model size increases. Evaluation awareness is the ability of the model to differentiate if it is being evaluated or is deployed.
This will have serious concern if not countered. Your feedback would be highly appreciated on the article.
Authors: Maheep Chaudhary, Ian Su, Nikhil Hooda, Nishith Shankar, Julian Tan, Kevin Zhu, Ryan Laggasse, Vasu Sharma, Ashwinee Panda
Large language models (LLMs) can internally distinguish between evaluation and deployment contexts, a behaviour known as evaluation awareness. This raises significant concerns for AI safety, as models may behave differently between testing and real-world implementation. Prior work has demonstrated this behaviour in a single, large-scale open-source model (70B parameters), leaving room for... (read 1740 more words →)