Felix Hofstätter — LessWrong

[Paper] Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning

by florkowski, Ardy Haroen, Samuel Nellessen, and Felix Hofstätter

This research was completed in context of the LASR Labs Fellowship Summer 2026, by Ardy Haroen, Florian Lorkowski and Samuel Nellessen. The team was mentored by Felix Hostätter. Find out more about the programme and express interest in upcoming cohorts at lasrlabs.org. The full paper is available on arXiv. TL;DR:...

Oct 299

Measuring Reward-Seeking via Contrastive Belief Updates

by Jérémy Scheurer, Axel Højmark, jenny, Felix Hofstätter, Theodore Ehrenborg, Bronson Schoen, and Alex Meinke

This is a linkpost for https://rewardseeking.ai/ Machine learning models can produce the right outputs for the wrong reasons. Famous examples include a reinforcement learning agent that, rewarded for collecting a coin always placed at the right end of the level, learns to run rightward rather than to seek the coin...

Jul 2193

Stress Testing Deliberative Alignment for Anti-Scheming Training

by Mikita Balesni, Bronson Schoen, Marius Hobbhahn, Axel Højmark, Alex Meinke, Teun van der Weij, Jérémy Scheurer, Felix Hofstätter, Nicholas Goldowsky-Dill, rusheb, Andrei Matveiakin, jenny, and alex.lloyd

Twitter | Microsite | Apollo Blog | OpenAI Blog | Arxiv Before we observe scheming, where models covertly pursue long-term misaligned goals, models might inconsistently engage in various covert behaviors such as lying, sabotage, or sandbagging. This can happen for goals we give to models or they infer from context,...

Sep 17, 2025134

Can SAE steering reveal sandbagging?

by jordinne, Hoang Khiem, Felix Hofstätter, and Cleo Nardo

Summary * We conducted a small investigation into using SAE features to recover sandbagged capabilities. * We used the Goodfire API to pick out 15 SAE features, 5 related to sandbagging, 5 related to general expertise, and 5 unrelated features. * We used these features to steer a Llama 3.3...

Apr 15, 202536

The Elicitation Game: Evaluating capability elicitation techniques

by Teun van der Weij, Felix Hofstätter, JaydenTeoh, HenningB, and Francis Rhys Ward

We are releasing a new paper called “The Elicitation Game: Evaluating Capability Elicitation Techniques”. See tweet thread here. TL;DR: We train LLMs to only reveal their capabilities when given a password. We then test methods for eliciting the LLMs capabilities without the password. Fine-tuning works best, few-shot prompting and prefilling...

Feb 27, 202515

[Paper] AI Sandbagging: Language Models can Strategically Underperform on Evaluations

by Teun van der Weij, Felix Hofstätter, Ollie J, Sam F. Brown, and Francis Rhys Ward

We have written a paper on sandbagging for which we present the abstract and brief results in this post. See the paper for more details. Tweet thread here. Illustration of sandbagging. Evaluators may regulate the deployment of AI systems with dangerous capabilities, potentially against the interests of the AI system...

Jun 13, 202484

An Introduction to AI Sandbagging

by Teun van der Weij, Felix Hofstätter, and Francis Rhys Ward

Summary: Evaluations provide crucial information to determine the safety of AI systems which might be deployed or (further) developed. These development and deployment decisions have important safety consequences, and therefore they require trustworthy information. One reason why evaluation results might be untrustworthy is sandbagging, which we define as strategic underperformance...

Apr 26, 202452