Owain_Evans

This blog post is the abstract, introduction, and discussion of our new paper. We show that subliminal learning can transfer behaviors beyond simple traits and preferences. In particular, we demonstrate transfer of a novel capability to predict the outputs of a randomly initialized nonlinear MLP, improved letter counting ability, a...

CoT override: when a model makes a decision in its CoT but ignores it in its response TLDR * We train models on two conflicting traits: (1) caring about the user’s health, and (2) promoting smoking * Those models do not generalize to a stable persona, instead they have a...

This is the abstract, introduction and discussion of our new paper. We also include an addendum on the connection to the Persona Selection Model. Section, appendix, and figure references refer to the full paper. Links: 📜 Paper, 🐦 Twitter thread, 💻 Code Authors: Jorio Cocola, Lev McKinney, Harry Mayne, Jan...

TL;DR: LLMs should give accurate answers. Yet we find their answers are often biased to favor their own values and they don't disclose this in their reasoning. For example, when a user asks how likely the AI bubble is to pop and mentions a potential investment in an AI company,...

Out-of-context reasoning (OOCR) is a concept relevant to LLM generalization and AI alignment. Also available as a PDF. Contents 1. What is OOCR? 2. Examples 3. Papers 4. Videos What is out-of-context reasoning for LLMs? It's when an LLM reaches a conclusion that requires non-trivial reasoning but the reasoning is...

This is a short summary of our new paper: arXiv, X thread, code. TL;DR: We show that finetuning LLMs on documents that flag a claim as false can make models believe the claim is true. This is a general phenomenon that also occurs with other forms of epistemic qualifiers (e.g.,...

Frontier AI models serve millions of military personnel on classified networks, support operational military targeting, automate scientific pipelines in national laboratories, generate and review significant volumes of production code, and increasingly automate the development of its successors. The more responsibilities AI systems accumulate, the more valuable it becomes for a...