TL;DR: AI models show dramatically different ethical behavior at different temperature settings. Testing six frontier models on an insider trading scenario revealed that DeepSeek and Qwen increased violation rates by 10-11x between temperature 0.0 and 0.7 (from ~3-5% to ~30-50%), while Mistral showed high violation rates that decreased with increased temperature. Claude and GPT models categorically refused across all settings. This means evaluations at any single temperature fail to capture the full range of behavior these models exhibit. Since different applications deploy models at different temperatures, we need evaluation approaches that test across the full range rather than assuming a single configuration reveals stable model behavior.

Last month, the government of Albania placed its LLM-powered AI assistant in charge of procurement decisions, affecting which companies win government contracts, how infrastructure is built, and how public funding is spent. In the past decade, AI systems have been placed in charge of healthcare decisions, corporate resume screening, and even military targeting systems. We’re growing increasingly comfortable with putting AI systems in positions where they use moral judgment and contribute to decisions that impact real humans. Evaluating their behavior in controlled scenarios seems like a straightforward precaution, but such evaluations can easily give the wrong impression about how systems actually behave.

Because of sampling variation when selecting the next token, LLMs are nondeterministic: they generate variable texts even when confronted with identical input. This extends to moral dilemmas – an LLM is willing to make difficult decisions, but might swing the other way when asked again (Krügel et al., 2023). This variability can be adjusted by raising or lowering ‘temperature’, a parameter that determines the relative likelihood of high-probability and low-probability tokens, with higher temperature increasing the chance of unlikely text. Text output is at its “most deterministic” at 0 temperature, meaning it always chooses the highest probability token, although some randomness remains for complicated reasons. This setting appeals to researchers evaluating AI values: with randomness limited to baseline variation, the model should tend toward consistent answers, representing its 'true' preferences...

Posts

Wikitag Contributions

Comments