Links: Paper | Code TL; DR. * Models condition their behavior on extraneous input features in undesirable ways — for example, on evaluation-likeness (resulting in evaluation gaming), or on the user's preferred answer (resulting in sycophancy). * Consistency training teaches a model to behave the same whether or not an...
This is a short write-up of work conducted as part of the MATS 9.0 program. Thanks to Victoria Krakovna for mentorship and Fred Bruford for research management. TL;DR: We introduce a pipeline that generates environment blueprints for realistic scheming propensity evaluations. As a case study, we test how well these...
TL;DR * To understanding the conditions under which LLM agents engage in scheming behavior, we develop a framework that decomposes the decision to scheme into agent factors (model, system prompt, tool access) and environmental factors (stakes, oversight, outcome influence) * We systematically vary these factors in four realistic settings, each...
This is an interim report produced as part of the Summer 2025 LASR Labs cohort, supervised by David Lindner. For the full version, see our paper on the LASR website. As this is an ongoing project, we would like to use this post both as an update, and as a...