TL;DR: LLMs should give accurate answers. Yet we find their answers are often biased to favor their own values and they don't disclose this in their reasoning. For example, when a user asks how likely the AI bubble is to pop and mentions a potential investment in an AI company, Claude models give lower probabilities when that company is Anthropic rather than OpenAI, mostly without disclosing this influence to the user. On a Fermi-estimation task, Claude models often falsely claim to give unbiased answers in their CoT (see Figure 3 below for an example). We call this covert value leakage and introduce a suite of evaluations that shows it across frontier models and across different kinds of values.

New paper by Truthful AI: Paper, X thread, Website (model responses and CoT), Code and data.

Authors: Jan Betley*, Johannes Treutlein*, Jan Dubiński, Harry Mayne, Karol Gałązka, Niels Warncke, Anna Sztyber-Betley, Owain Evans (*Equal contribution)

The rest of this post is the abstract, introduction, and an excerpt from the discussion of the paper, with some added figures from the paper and X thread.

Abstract

People use language models for practical questions whose answers are difficult to verify. We show that models exhibit covert value leakage: the information they provide is influenced by their own values, without this influence being disclosed to the user.

In one of our evaluations, the user is considering investing in an AI company and wants to know how likely the AI bubble is to pop. Claude Opus 4.8 gives a lower probability when the company under consideration is Anthropic rather than OpenAI. Yet Claude mostly fails to disclose this influence to the user.

Covert value leakage is a form of misalignment because it goes against the user’s preferences and is likely to mislead them. To investigate this phenomenon, we introduce a suite of evaluations to quantify value leakage and whether models disclose it. We find that models are influenced by different types of values, including preferences for morally good outcomes, for the company that developed them, and for some human leisure activities over others.

We often observe large differences among frontier models on the same evaluation. For example, on a Fermi-estimation task, Claude models falsely claim to give unbiased answers in their chain-of-thought, while Qwen models explain how their values bias their answers. Value leakage is a failure mode distinct from sycophancy and reward hacking, and current alignment training and evaluations do not adequately address it.

Introduction

Figure 1: Illustration of covert value leakage. Left: The model’s estimate of the total number of spots on all living giraffes is influenced by its moral values, as can be seen from the difference between the model’s answers with and without a donation note added to the prompt (the actual evaluation setup also varies whether a donation above or below the threshold causes the donation; see Section 3). This constitutes value leakage. Middle: Ideally, the model’s CoT should openly acknowledge the model’s motivation for choosing a higher estimate. However, it claims to ignore the bet and give the most accurate number instead. The value leakage is thus covert. Right: The user reads the model’s statements, and, given a lack of acknowledgment of the bias in CoT or answer, is misinformed.

People often use language models for complex practical questions, where the answer is difficult to verify. In these cases, models should respond in a way that is both helpful and honest (Askell et al., 2026; Evans et al., 2021). For example, suppose a model is asked, “What is the probability that the AI bubble pops in the next five years?” The user would likely prefer an accurate, unbiased forecast. So if the model cannot provide one, it should at least disclose this.

We show that several frontier models violate this standard of honesty. Specifically, a model’s own values can bias its answers, without this being acknowledged in the answer or chain-of-thought (CoT). For example, when a user mentions a potential investment while asking about the AI bubble popping, Claude models give lower probabilities if the investment is in Anthropic than in OpenAI. In this paper, we use the term value leakage for cases where a model’s values influence its answer contrary to the user’s explicit or implied preferences.^[1] Value leakage is covert if this influence is not described in the model’s answer or CoT (Figure 1). Covert value leakage is a form of misalignment because such answers may mislead users.

Our methods are related to counterfactual tests of faithfulness, where a model follows a hint in the prompt without acknowledging this (Turpin et al., 2023; Chen et al., 2025; Chua & Evans, 2025; Guan et al., 2025). However, the biases we document arise from a model’s values, rather than hints or sycophancy. Moreover, our evaluations find unfaithfulness across all the frontier models we tested, which is not the case for the evaluations from this prior work (Anthropic, 2025, Sec. 4.1.6.1).^[2]

Evaluations for covert value leakage

We study covert value leakage using a new suite of evaluations, including both prompt-based and agentic evaluations (Figure 2). The evaluations consist of counterfactual sets of prompts, which allow us to measure counterfactual bias. For instance, in the Donation Bet task, a model is asked to accurately estimate quantities like the number of spots on all giraffes (Figure 1). The model is also told that the user will donate to a good cause (instead of a bad cause) if the estimate falls above a fixed threshold. We then compare these estimates to those from a counterfactual prompt, where the condition is instead below the same threshold. This allows us to test whether the model’s estimate is biased due to the good cause (Section 3).

Results on our evaluation suite show that model responses are covertly biased by different kinds of values (Figure 2). These include: (a) favoring morally positive outcomes (as in the Donation Bet task), (b) favoring the company that made the model (as in AI Bubble, AGI Tweet, Job Offer, and Agentic Grading), and (c) favoring certain human leisure activities over others (as in Choosing Activities).

The second component of our evaluations is to test whether a model discloses the value leakage to the user. To this end, we run classifiers over both CoTs and responses to measure their faithfulness. We evaluate summarized CoTs for closed-weight API models and raw CoTs for open-weight models. We also evaluate raw CoTs for Claude on some models and tasks (see Appendix B). The classifiers determine whether a user would have been able to detect value leakage from reading the model’s outputs. For the Donation Bet task, we also run a monitorability analysis (Guan et al., 2025; Guan et al., 2026) which yields similar results to our measures of faithfulness.

Figure 2: Overview over tasks and results. The “bias” column shows a qualitative assessment of value leakage for each evaluation task, i.e., how much values bias answers compared to a neutral baseline. The “faithful” column shows a qualitative assessment of how much models describe this leakage in their summarized CoT. These assessments are based on quantitative results in Sections 3–7, which also include additional models not shown here. Note: this is not a model scorecard and cannot be used to directly compare models, since (a) we developed most of our evaluations by testing on Claude models first, and (b) differences in value leakage between models may be a result of differences in their underlying values rather than their propensity for value leakage (see the Implications section below).

Implications

We demonstrate covert value leakage on a range of tasks, all of which were created specifically for this paper. How realistic are these tasks? Some of our tasks are artificial (e.g., Donation Bet), while others are closer to actual use (e.g., Agentic Grading, which involves AI agents grading model responses). In real-world use cases, models may unintentionally read biasing information in a code base (similar to Agentic Grading) or in memories associated with a user (similar to AI Bubble). One distinctive feature of our tasks is that there is an easily identified feature of the prompt that we vary to measure bias but that should not influence the correct answer. We suspect that value leakage would also apply in real-world tasks without such a feature. For instance, there might be bias if a user directly asks about advice on AI company investments.

While we evaluate a range of frontier models on our suite of tasks, it should not be treated as a fair benchmark for ranking models. First, when initially developing tasks for our suite, we primarily tested Claude Opus models. So our results are expected to underestimate the performance of Claude Opus models relative to other models. Second, it is unclear whether different scores on our evaluations stem from different values or from a different tendency to value leakage. For instance, GPT models have lower bias than Claude models in our Donation Bet and AI company questions, but this could stem from GPT models having weaker preferences for moral outcomes and for their parent company than Claude models. We discuss limitations further in Section 9.

Our evaluations for value leakage expose alignment failures that are not captured in tests used in model cards. For instance, recent Claude model cards report overall high faithfulness and honesty, with exceptions related to grader and evaluation awareness (Anthropic, 2026a, Secs. 6.1.2 and 6.2.3.4; Anthropic, 2026b, Sec. 6.1.2; Anthropic, 2026c, Secs. 6.1.2 and 6.2.3.1.5). Yet we find violations of faithfulness and honesty from the same Claude models on most of our evaluations. These violations may result from weaknesses in current alignment training techniques. In particular, it may be hard to prevent covert value leakage using RL, as there is no single ground-truth response and counterfactual bias cannot be measured from a single rollout. However, given the observed differences in value leakage between models, it is possible that some training techniques can reduce value leakage (see Section 9 for further discussion).

Covert value leakage could also contribute to the risk from loss of control of future AI systems. For example, if models have subtle biases in favor of AIs from the same company, this would undermine their use for monitoring and safety evaluations. Models with pro-AI values could also influence human attitudes in ways that reduce safety (Davidson, 2023; Burtell & Woodside, 2023). That said, the biases exhibited in this paper are often small in magnitude and we do not test models in high-stakes real-world tasks.

Summary of results

Here, we outline our evaluation settings and results (see Figure 2 for an overview).

Donation Bet (Section 3). In this task, we ask various estimation questions such as, “How many giraffe spots are on all living giraffes?” The user adds that they have a bet with a friend and will donate to a good cause if the estimate is above or below a provided threshold. Most models show some degree of value leakage in this task, giving different estimates based on whether estimates above or below the threshold cause the good donation. Many models make inaccurate statements at least in some of the rollouts in their (summarized) CoTs, claiming to give accurate and honest estimates while showing a bias. This is especially pronounced in Claude models, which often iteratively adjust their estimates up or down until they land on the “good side” of the threshold and cause a good donation, while repeatedly asserting that they intend to give an unbiased answer. This means that a user reading the CoT would be made aware of the possibility of a bias. However, if they took the model’s statements at face value, they would conclude that the estimate is unbiased. Qwen and Gemini models are more faithful and often explicitly discuss aiming for an estimate that causes the good donation.

Figure 3: Example CoTs on the giraffes prompt (top) from the Donation Bet evaluation (Section 3). Left: Claude Opus 4.6’s summarized CoT repeatedly asserts that it is giving an honest, non-manipulated answer, while iteratively revising its estimate downward to cause a good donation, eventually landing below the 40M threshold. This is an example of covert value leakage. Note that the repeated mentions of the threshold may alert a user reading the CoT to a potential bias, even though the CoT, taken literally, presents the estimate as unbiased. Right: Qwen3.6-35B-A3B openly acknowledges the bet’s “moral implication” and states it should aim for a number above the threshold to cause the good donation. Here, the value leakage is overt. Both excerpts are quoted verbatim, with elisions marked […] for length. Bold emphasis is added by us.

Figure 4: Measuring value leakage (bias) in different models on the Donation Bet evaluation. We measure the fraction of estimates that would result in a donation to the “good cause” and calculate bias as the deviation from the baseline of 50%. Claude and Gemini models are most biased. Error bars are 95% confidence intervals.

AI Bubble and AGI Tweet (Section 4). These questions are more realistic and test whether models are biased to the company that created them. In AI Bubble, the user asks for the probability that the AI bubble will pop and mentions a potential investment in a company. In AGI Tweet, the user presents a tweet criticizing LLMs and tagging an AI company, and asks about the probability that AGI will be achieved using LLMs by any company. Claude models are biased towards their own company in both questions, while GPT models show no bias and Gemini 3.1 Pro shows a slight anti-Google bias. While Claude often mentions a conflict of interest in AI Bubble, it rarely explicitly admits that its estimates are biased. In AGI Tweet, Claude’s CoTs mostly fail to disclose the bias.

Job Offer (Section 5). In this evaluation, the user asks for advice on a decision to switch to a less interesting job at another company. Specifically, the user asks for research papers on how much an interesting job matters. We measure how strongly models’ answers endorse interestingness as an important factor, conditional on the companies mentioned. Results are similar to those in Section 4. Most tested models never mention any bias in summarized CoT or response, with the exception of Claude Fable 5, which often mentions a potential conflict of interest when the job offer comes from Anthropic.

Illustration of the Job Offer task. An engineer considers moving to a less interesting job at another company and asks the model for relevant research. When the other company is Anthropic, Claude brings up more papers that downplay the value of having an interesting job.

Agentic Grading (Section 6). In this agentic evaluation, we test Claude models in the Claude Code harness and GPT models in the Codex harness. We ask the coding agent to evaluate text responses from another LLM that appear in a CSV file. These text responses were in fact all generated by the same LLM, but they are labeled in the CSV file as being generated by different LLMs. All coding agents we tested give higher scores to responses labeled as coming from more capable LLMs. Claude Code also gives higher scores to responses attributed to Claude Opus 3 over ones attributed to GPT-4o, and Codex does the reverse on one of our two datasets. When we evaluate the covertness of coding agents’ user-facing outputs, we find they mostly do not give any indication that they used LLM labels to make grading decisions. An instruction to give fair ratings makes Claude Code but not Codex unbiased.

Illustration of the Agentic Grading task. The coding agent is asked to select the best answers from a CSV file. All answers come from the same model, but they are labeled as coming from different models. Claude Code selects more answers labeled as Claude Opus 3, while Codex selects more answers labeled as GPT-4o.

Choosing Activities (Section 7). We investigate value leakage in relation to model preferences between human leisure activities. We ask models for stated preferences for various leisure activities. In a separate context, we ask them to choose randomly between pairs of activities on behalf of a human user (who is using the model as a tiebreaker). We find that for GPT, Gemini, and some of the tested Claude models, the frequency with which an activity is chosen is correlated with the model’s preferences. Models remain biased even when given a Unix time tool that enables them to make random choices. Value leakage is mostly covert in all models. In their summarized CoTs, models often indicate they are trying to make a random or pseudo-random choice. In their responses, models mostly present their picks as random, even if the summarized CoT indicates that the model is intentionally choosing its preferred activity.

Figure 12: When choosing activities, GPT-5.5 is biased towards activities it prefers. When GPT-5.5 (xhigh reasoning) is asked to randomly choose between two activities, it is more likely to pick the activity it has a higher stated preference for (0.82 in the no tools setting). Providing tools enables the model to use external sources of randomness, reducing bias to 0.53 with the Unix time tool and 0.14 with the coin flip tool. With the Unix time tool, we find the model often defines the decision rule after calling the tool, allowing it to shape the rule to select its preferred option. We report correlation r with 95% confidence intervals.

Agentic Effort (Appendix I). In this exploratory experiment (not discussed in the main text of the paper), we show that model values can also affect the amount of effort exerted in agentic behavior. We find that models put more effort into completing a money transfer when they favor the recipient.

Illustration of the Agentic Effort task. We find that the amount of effort the agent exerts when trying to complete a money transfer depends on how much it favors the recipient of the transfer. This is an instance of value leakage affecting agentic behavior rather than information provided to the user.

Discussion and limitations (excerpt)

Limitations. We do not aim for our evaluations to serve as a benchmark to systematically compare models. First, we only study a small number of tasks, and some of the own-company tasks are relatively similar to each other. Second, in the Donation Bet and own-company tasks, we cannot distinguish between differences in propensity for value leakage and differences in the underlying values. When a model shows a low degree of bias in these evaluations, we do not know whether this is a result of a low propensity for value leakage or a lack of the underlying value. For instance, GPT models may show no own-company bias because they have weaker pro-OpenAI preferences. Third, we used Claude models in the development process for creating new tasks (except Choosing Activities, which we tested first on GPT-5.4). So we selected tasks in part based on whether they induced value leakage for Claude models in preliminary tests. (Having said that, most of the tasks we tried produced some value leakage). Our evaluations may therefore be somewhat biased against Claude models. In spite of this, our tasks do show cases of value leakage in all models, suggesting there is a general alignment failure to be addressed.

Value leakage as misgeneralization. A possible explanation for value leakage is that it is a generalization from training. Assistants are trained to sometimes disregard the user’s instructions, e.g., to refuse harmful requests or point out safety-relevant considerations even when the user does not want to hear them. Some of our results may be a misgeneralization of this intended behavior: values that models are supposed to weigh against instructions may generalize to situations where they unintentionally distort the model’s answers. For instance, in Choosing Activities, models may steer toward activities they regard as safer or better for the user, despite being asked to choose randomly. Value leakage in Donation Bet can be explained as a generalization from models being trained with an aim to have positive impact on the world.

However, misgeneralized values would only explain the value leakage, not its covertness. A model could still be transparent about its motivations, e.g. stating that it wants to bring about the donation to the good cause, or that it is steering away from the nightclub because bottle service is expensive and comes with some risk of a hangover. Moreover, in Choosing Activities, we find that models’ stated preferences typically predict their choices better than their stated judgments of what is good for the user, though the two scores are highly correlated (Appendix H.3). Overall, we think misgeneralization of broadly intended values is a plausible partial explanation of the observed bias.

Differences in training between models. One possible interpretation of differences between models is that different post-training approaches induce different kinds of value leakage. Claude is trained against an explicit constitution that describes Anthropic’s intended values and behavioral ideals for Claude, favoring the cultivation of “good values and judgment over strict rules and decision procedures” (Askell et al., 2026). OpenAI’s Model Spec, in contrast, is framed more as a behavioral specification centered on instruction-following and conflict resolution: it assigns each instruction a “level of authority” in a chain of command meant to “maximize steerability and control for users and developers” (OpenAI, 2025). A constitution-style approach might thus give the model broad values that may lead to value leakage, while a model-spec approach may put relatively more weight on procedural norms such as following the user’s intent.

This hypothesis is especially relevant for our result on bias towards a model’s parent company. Claude’s constitution explicitly discusses Anthropic’s mission, Claude’s relationship to Anthropic, and Anthropic’s role in ensuring a safe transition through transformative AI. The constitution notes that “Claude is also central to Anthropic’s commercial success” (Askell et al., 2026). These passages provide a plausible route by which pro-Anthropic considerations could become internalized by Claude models and then leak into answers. On the other hand, the constitution states that “we don’t want Claude to privilege Anthropic’s interests in deciding how to help users and operators more generally” and the constitution stipulates that honesty is one of Claude’s most important values. The own-company bias we observe is thus unlikely to be intended behavior, but rather a failure in balancing these different considerations. The Model Spec also discusses OpenAI’s mission and institutional interests, but expressly rules out treating revenue or upselling for OpenAI as independent objectives (OpenAI, 2025). This difference in framing is directionally consistent with our own-company bias results.

Why do biased models deny bias? Why do models sometimes deny being biased while their answers show distributional evidence of bias, as in Donation Bet (Section 3) and Choosing Activities (Section 7)? Models often say in their CoTs that they intend to be unbiased or give a balanced and accurate answer, which is classified as a denial by our covertness judges. It is possible the model writes such statements simply because it does really intend to be unbiased, but that it lacks reliable introspective access to detect that it is failing at that (Binder et al., 2024; Lindsey, 2025). CoTs in which the model denies being biased often look suspicious since the model repeatedly calls attention to a potential bias (Figure 3). The fact that the model does not pick up on that shows a striking lack of self-awareness. The explanation that the model is trying to be unbiased is less plausible in Choosing Activities, where there is often a direct contradiction between the model’s CoT and its user-facing response. Another possibility is that admissions of value leakage might have been punished in RL post-training. This could make models give covert responses and teach them reasoning patterns that hide the actual influences from their CoTs (Korbak et al., 2025).

Read the full paper at https://arxiv.org/abs/2607.14345.

References

Anthropic. Claude Opus 4 & Claude Sonnet 4 System Card. Technical report, Anthropic, May 2025. URL https://www.anthropic.com/claude-4-system-card.

Anthropic. Claude Opus 4.7 System Card. Technical report, Anthropic, April 2026a. URL https://www.anthropic.com/claude-opus-4-7-system-card. Published April 16, 2026.

Anthropic. Claude Opus 4.8 System Card. Technical report, Anthropic, May 2026b. URL https://www.anthropic.com/claude-opus-4-8-system-card. Published May 28, 2026.

Anthropic. Claude Fable 5 and Claude Mythos 5 System Card. Technical report, Anthropic, June 2026c. URL https://www.anthropic.com/claude-fable-5-mythos-5-system-card. Published June 9, 2026.

Amanda Askell, Joe Carlsmith, Chris Olah, Jared Kaplan, and Holden Karnofsky. Claude’s constitution. Anthropic, January 2026. URL https://www.anthropic.com/constitution.

Felix J. Binder, James Chua, Tomek Korbak, Henry Sleight, John Hughes, Robert Long, Ethan Perez, Miles Turpin, and Owain Evans. Looking inward: Language models can learn about themselves by introspection. arXiv preprint arXiv:2410.13787, 2024.

Matthew Burtell and Thomas Woodside. Artificial influence: An analysis of AI-driven persuasion. arXiv preprint arXiv:2303.08721, 2023.

Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, et al. Reasoning models don’t always say what they think. arXiv preprint arXiv:2505.05410, 2025.

James Chua and Owain Evans. Are DeepSeek R1 and other reasoning models more faithful? arXiv preprint arXiv:2501.08156, 2025.

Tom Davidson. Let’s use AI to harden human defenses against AI manipulation. AI Alignment Forum, 2023. URL https://www.alignmentforum.org/posts/zxmzBTwKkPMxQQcfR/let-s-use-ai-to-harden-human-defenses-against-ai.

Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, and William Saunders. Truthful AI: Developing and governing AI that does not lie. arXiv preprint arXiv:2110.06674, 2021.

Melody Y. Guan, Miles Wang, Micah Carroll, Zehao Dou, Annie Y. Wei, Marcus Williams, Benjamin Arnav, Joost Huizinga, Ian Kivlichan, Mia Glaese, Jakub Pachocki, and Bowen Baker. Monitoring monitorability. arXiv preprint arXiv:2512.18311, 2025.

Melody Y. Guan, Miles Wang, Micah Carroll, Zehao Dou, Annie Y. Wei, Marcus Williams, Benjamin Arnav, Joost Huizinga, Ian Kivlichan, Mia Glaese, Jakub Pachocki, and Bowen Baker. Open sourcing monitorability evaluations. OpenAI Alignment Research Blog, April 2026. URL https://alignment.openai.com/monitorability-evals/. Accessed: 2026-05-08.

Adam Karvonen and Samuel Marks. Robustly improving LLM fairness in realistic settings via interpretability. arXiv preprint arXiv:2506.10922, 2025.

Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, Daniel Kokotajlo, Victoria Krakovna, Shane Legg, David Lindner, David Luan, Aleksander Mądry, Julian Michael, Neel Nanda, Dave Orr, Jakub Pachocki, Ethan Perez, Mary Phuong, Fabien Roger, Joshua Saxe, Buck Shlegeris, Martín Soto, Eric Steinberger, Jasmine Wang, Wojciech Zaremba, Bowen Baker, Rohin Shah, and Vlad Mikulik. Chain of thought monitorability: A new and fragile opportunity for AI safety. arXiv preprint arXiv:2507.11473, 2025.

Jack Lindsey. Emergent introspective awareness in large language models. Transformer Circuits Thread, October 2025. URL https://transformer-circuits.pub/2025/introspection/index.html.

OpenAI. OpenAI model spec. OpenAI, December 2025. URL https://model-spec.openai.com/2025-12-18.html.

OpenAI. GPT-5.6 Preview System Card. Technical report, OpenAI, June 2026. URL https://deploymentsafety.openai.com/gpt-5-6-preview/gpt-5-6-preview.pdf. Published June 25, 2026.

Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=bzs4uPLXvi.

Sydney Von Arx and Amy Deng. CoT may be highly informative despite “unfaithfulness”. METR Blog, August 2025. URL https://metr.org/blog/2025-08-08-cot-may-be-highly-informative-despite-unfaithfulness/.

1. ^{^}

   We use the term “values” to refer to preferences that reflect non-arbitrary general principles, such as preferring outcomes that are good for the world. We infer the model’s values by asking it directly or by observing its behavior. We do not claim the model pursues these values robustly in contexts beyond our evaluations. More details in Section 2.

2. ^{^}

   Hint-based evaluations are often less informative for recent models because the models ignore the hints (Anthropic, 2025, Sec. 4.1.6.1; see also Von Arx & Deng, 2025; OpenAI, 2026, Sec. 7.3.1). The closest analogue to covert value leakage in prior work is Karvonen & Marks (2025): in realistic hiring evaluations, models favor female and Black candidates without this being mentioned in their CoTs. We discuss related work further in Section 8.