Koby Lewis — LessWrong

Are Persona Features Coherent?

Epistemic status: would not be surprised if I'm missing prior discussion that could shed light on this. Let me know if you know of anything! What is the difference between a "deception feature" and a "deceptive persona feature"? As far as I can tell, the working definitions in the canonical...

Oct 2711

Can the Assistant Axis Predict Reward-Hacking?

NOTE: This article was written by Jansen Nye, a member of my research lab based on some exploratory studies we've done. As he notes, we collaborated on this research. The article was written by him with some edits suggested by me. -- Koby TL;DR The Assistant Axis doesn't seem to...

Oct 2619

You Should Think About What Needs to Go Right + My List

When people ask me what I do all day, I usually start by saying "there are a lot of ways AI can go wrong". I believe this, but recently I've got to thinking that just "prevent the bad thing from happening" isn't very likely to lead to a world I...

Aug 3116

"Correct Answer Features" Cannot Explain Multiple Choice Capabilities

TL;DR: I present theoretical and empirical evidence that LLMs cannot be (exclusively) using a "correct answer feature" as the main mechanism by which they perform multiple choice question answering. A hypothetical correct-answer feature would indicate the "correctness" of an option on the final token(s) of that option. However, such a...

Jun 308

Porting MACHIAVELLI To Inspect

TL;DR The MACHIAVELLI benchmark aims to measure how often AI agents take unethical actions when pursuing a goal. Because this is an alignment benchmark, not a capabilities benchmark, it is more important that it be run on each new generation of AI. By re-implementing MACHIAVELLI using the Inspect framework, I...

Jun 179

A Call for Better Type Hints in AI Safety Tooling

Good type hints lead to code that is more maintainable, is easier to understand, and has fewer bugs. If you'd like a quick, general intro on why, see this article, but suffice it to say that types give us a way to automatically check assumptions and invariants[1]. There are ways...

May 2813

What Do You Mean by a Two-Year AGI Timeline?

Until recently, I was a bit confused about what people meant when they talked about AGI[1] timelines. My hope in writing this short note is to help clarify things for anyone in a similar position. In casual conversation, people often say something like "I have a two-year AGI timeline", without...

Apr 306