Cross-posted from our recent paper: "But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectors" : https://arxiv.org/abs/2505.17760

Code available at: https://github.com/watermeleon/judge_with_steered_response

TL;DR: We use steering vectors to generate more honest versions of an LLM response, helping LLM judges detect subtle forms of dishonesty like sycophancy and manipulation that they normally miss. We also introduce a new dataset with prompts designed to provoke subtle manipulation.

Abstract

A fundamental challenge in AI alignment is that as systems become more capable, they may learn subtle forms of deception that evaluators (humans or other LLMs) struggle to detect. Even when an AI's response is technically accurate, it might be manipulative, sycophantic, or misleading in ways that optimize for human approval rather than truth. Moreover, most honesty benchmarks focus exclusively on factual knowledge or explicitly harmful behavior and rely on external LLM judges, which are often unable to detect less obvious forms of dishonesty.

In this work, we introduce a new framework, Judge Using Safety-Steered Alternatives (JUSSA), which utilizes steering vectors trained on a single sample to elicit more honest responses from models, helping LLM judges in the detection of dishonest behavior. To test our framework, we introduce a new manipulation dataset with prompts specifically designed to elicit deceptive responses. We find that JUSSA enables LLM judges to better differentiate between dishonest and benign responses, and helps them identify subtle instances of manipulative behavior.

Motivation:

We build upon recent work by Dunefsky et al. (2025), which showed that steering vectors trained on a single example can generalize well for various safety-related behaviors. While steering for safety training shows promise, models may eventually learn to work around such interventions. Therefore, we consider whether steering vectors may be valuable as tools for improving evaluations of models instead of simply directly intervening on model behavior.

Moreover, subtle forms of dishonesty—like sycophancy, selective evidence presentation, or emotional manipulation—pose a particularly difficult challenge. These behaviors may involve misleading but factually correct statements, making them difficult for standard evaluation methods to detect, yet pote...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments