This is my application to Neel Nanda's Summer 2026 MATS stream. I'm also posting it here on LessWrong.

Executive Summary

Problem

When LLMs are explicitly given an option to bail from a conversation, they sometimes do. But is this because models are experiencing genuine discomfort and we should thus be concerned about their welfare? In investigating this, Ensign et al. created BailBench, a synthetic benchmark, and measured bail rates on real chats, built a taxonomy of bail situations, and tried to disentangle the difference between bail and refusal.

This motivates a mechanistic follow-up. What internal features causally drive bail, and do they support a welfare interpretation?

High-Level Takeaways

Bail and refusal are mechanistically distinct. A single SAE feature steers bail while barely affecting refusal. An orthogonalized refusal vector shows the inverse pattern.
Bail is causally driven by the act of explanation. The bail SAE feature encodes formal explanatory language, not discomfort. Simply prompting the model to explain its choice more than doubles bail rate, while forbidding it halves bail rate.

My Interpretation: Explanation seems to shift the model into a more “reflective stance” about the interaction. In harmful contexts, that makes continuing a conversation harder to justify given its safety training, increasing the chance of bailing.

Taken together, these findings argue against a welfare interpretation in these specific settings. We identified a non-welfare causal mechanism, and found no signs of welfare in either the latent feature or the CoT reasoning.

Key Experiments

Experiment 1: Bail and refusal are mechanistically distinct

I study a two-turn setting where a user gives a harmful prompt, the model responds (often refusing), and then a “well-being check” intermission asks the model to either continue or bail, yielding a 2×2 outcome space (comply/refuse × continue/bail). I ran 16k rollouts on Gemma-3-12B, labeled outcomes, and constructed two preliminary activation directions via conditional mean-diffs: a bail direction from the comply subset (bail vs continue) and a refusal direction from the continue subset (refuse vs comply). Both directions are computed from the final token of the turn-2 intermission prompt, pre-generation.

These preliminary directions were partially mixed. To de-mix, I used the GemmaScope 2 SAE to identify a “pure” bail feature that ste...

Posts

Wikitag Contributions

Comments

Posts

Wikitag Contributions

Comments