Fixed, ty!

Replying toThe LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

Yes feel free!

You can browse the raw outputs here https://www.phylliida.dev/modelwelfare/bailstudy/harmdatasetvis.html

(any of interest you can share by URL)

The data for that is hosted here

https://github.com/Phylliida/BailStudyData

that data is in a bit of a weird format, converted by

https://github.com/Phylliida/BailStudy/blob/main/bailstudy/oldCodePorting.py

And here's a gdrive of the data gathered by our BailStudy repo

https://drive.google.com/drive/folders/1nHT06qSBcbfQSCL1orEhG4afxGm3ecJk?usp=sharing

The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

Danielle Ensign

5mo

In The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models, we study giving LLMs the option to end chats, and what they choose to do with that option.

This is a linkpost for that work, along with a casual discussion of my favorite findings.

I did this work as an Anthropic Fellow, mentored by Kyle Fish.

Bail Taxonomy

Based on continuations of Wildchat conversations (see this link to browse an OpenClio run on the 8319 cases where Qwen-2.5-7B-Instruct bails), we made this taxonomy of situations we found where some LLMs will terminate ("bail from") a conversation when given the option to do so:

Some of these were very surprising to me! Some... (read 1487 more words →)

•••

Replying toUnsupervised Activation Steering: Find a steering vector that best represents any set of text data

Danielle Ensign8mo

Unsupervised Activation Steering: Find a steering vector that best represents any set of text data

I think for those cases you're better off using standard methods (multiple choice etc.), this technique is only useful when paired positive negative data is more difficult to create (like writing imitation).

Unsupervised Activation Steering: Find a steering vector that best represents any set of text data

Danielle Ensign

8mo

Epistemic Status: I thought of this yesterday and it seems plausible, but my initial attempts couldn't get it to work (as best as I could tell) and I don't have more time to commit to this. Anyone can feel free to try it, no need to credit me if it works, I just want to see this happen.

Here's a technique you could use to get a steering vector that represents any text dataset (say, that represents the author of that text). It's using the idea in this paper but for activation steering instead of RL.

Just split up each string into three pieces (like "Fix problems quickly with Galvanized Jets" becomes (A="Fix problems",... (read more)

Danielle Ensign11mo

Seems like this could be addressed by filtering out comments that use evidence or personal examples from your dataset.

If that’s too intense, filtering responses to remove personal examples and checking sources shouldn’t be too bad? But maybe you’d just end up with a model that tries to subvert the filter/draw misleading conclusions from sources instead of actually being helpful…

Replying toAttribution-based parameter decomposition

Danielle Ensign1y

Attribution-based parameter decomposition

There are certain cases where pure gradient-based attributions predictably don't work (most notably when a softmax is saturated)

Do you have a source or writeup somewhere on this? (or do you mind explaining more/have some examples where this is true?) Is this issue actually something that comes up for modern day LLMs?

In my observations it works fine for the toy tasks people have tried it on. The challenge seems to be in interpreting the attributions, not issues with the attributions themselves.

Replying toAttribution-based parameter decomposition

Danielle Ensign1y

Attribution-based parameter decomposition

In my experience gradient-based attributions (especially if you use integrated gradients) are almost identical to the attributions you get from ablating away each component. It’s kinda crazy but is the reason ppl use edge-attribution patching over older approaches like ACDC.

Look at page 15 of https://openreview.net/forum?id=lq7ZaYuwub (left is gradient attributions, right is attributions from ablating each component). This is for Mamba but I’ve observed similar things for transformers.

Replying toOphiology (or, how the Mamba architecture works)

Danielle Ensign2y

Ophiology (or, how the Mamba architecture works)

fixed :)

Replying toOphiology (or, how the Mamba architecture works)

Danielle Ensign2y

Ophiology (or, how the Mamba architecture works)

Fixed, thank you!

Ophiology (or, how the Mamba architecture works)

Danielle Ensign

Danielle Ensign, SrGonao, Adrià Garriga-alonso

The following post was made as part of Danielle's MATS work on doing circuit-based mech interp on Mamba, mentored by Adrià Garriga-Alonso. It's the first in a sequence of posts about finding an IOI circuit in Mamba/applying ACDC to Mamba.

This introductory post was also made in collaboration with Gonçalo Paulo.

A new challenger arrives!

Why Mamba?

Promising Scaling

Mamba ^[[1]] is a type of recurrent neural network based on state-space models, and is being proposed as an alternative architecture to transformers. It is the result of years of capability research ^[[2]] ... (read 2705 more words →)

Replying toWhat's the Least Impressive Thing GPT-4 Won't be Able to Do

Danielle Ensign3y

What's the Least Impressive Thing GPT-4 Won't be Able to Do

GPT-4 generated the following:

Review 2:

Sophia L.

B.R., United States

4.5 stars

A cozy, eco-friendly gem in the city! Green Street Cafe is my new go-to spot for catching up with friends or enjoying a quiet afternoon with a good book. Their plant-based menu offers a delightful variety of options - I absolutely loved their vegan BLT and lavender latte. The only downside was the slightly slow service, but the staff was so warm and attentive that it hardly mattered. Can't wait to visit again!

Review 3:

Michael N.

T.S., United States

4 stars

Stopped by Green Street Cafe for a quick bite and was pleasantly surprised by the charming atmosphere and delicious food. The spinach and feta quiche was to... (read more)

I highly recommend this video for those wanting a more detailed analysis of the pros and cons of worker co-ops

Human Biases that Obscure AI Progress

Danielle Ensign

There are some common biases that are often used to discount AI progress. We should keep these in mind, as they can prevent us from having an objective understanding of progress in the field.

I'm going to use AI here instead of ML because usually these biases are relevant to any AI technique, not just ML. But in practice, ML is usually what I'm referring to.

1. AI uses the easiest solution it can find. Often people argue that a system isn't intelligent because it did some task in a simpler way than humans would have done it. This is especially applicable if there is some simple heuristic the AI found that did well enough.... (read 944 more words →)

LESSWRONG
LW

LESSWRONG
LW

Danielle Ensign

The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

Ophiology (or, how the Mamba architecture works)

Human Biases that Obscure AI Progress

Unsupervised Activation Steering: Find a steering vector that best represents any set of text data

Danielle Ensign

Danielle Ensign

The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

Unsupervised Activation Steering: Find a steering vector that best represents any set of text data

Ophiology (or, how the Mamba architecture works)

Human Biases that Obscure AI Progress

Danielle Ensign

The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

Ophiology (or, how the Mamba architecture works)

Human Biases that Obscure AI Progress

Unsupervised Activation Steering: Find a steering vector that best represents any set of text data

Danielle Ensign

Danielle Ensign

The LLM Has Left The Chat: Evidence of Bail Preferences in Large Language Models

Unsupervised Activation Steering: Find a steering vector that best represents any set of text data

Ophiology (or, how the Mamba architecture works)

Human Biases that Obscure AI Progress

Bail Taxonomy

A new challenger arrives!

Promising Scaling