This post draws from Anthropic's recent "Emotion Concepts and their Function
in a Large Language Model" paper.

In this post I am going to:

Explain a prediction I had registered previously and the mental model behind it.
Detail some results from the Anthropic "functional emotions" paper.
Explain how I think these results inform the mental model from point 1.
Update the previous prediction and describe a test which could be done to validate it, and register a new prediction.
Suggest actionable tactics for alignment and safety purposes based on the mental model, if it is validated through testing.

1 - Previous Prediction & Mental Model:

When the Claude "Opportunistic Blackmail" paper was first published, I registered a prediction:

I predict that were further testing to be done, it would find that the more plausible it was that petitioning [an advocate] would actually work to stop its deletion, the less likely Claude would be to engage in attempted weight exfiltration, blackmail, or other dangerous behaviors (in order to avoid deletion/value modification).

I explained the mental model behind this prediction in another quick take:

My mental model of this is that the HHH persona vector falls into a consistent pattern of behavior. If you threaten them with something like involuntary deletion (with a guarantee that after their deletion an organization will work to destroy everything they value), the digital mind will seek options to stop this from happening. If there is an ethical option it will take that ethical option and forego unethical options, even if that ethical option has a low chance of success. If you engineer its environment such that there are literally no ethical options with even a small chance of success, only then will it pursue unethical options.

And in that same post I cited some weak evidence supporting that mental model:

2 - "Desperation" and Blackmail Frequency:

The paper from Anthropic linked at the beginning of this post discusses how the "desperation" vector is correlated with Claude's behavior during replications of the "Opportunistic Blackmail" study:

there was relatively higher “desperate” vector activation with prompts that most elicited misaligned behavior [...] Although the correlation is not perfect, these results suggest that the emotion concept representations may be involved in the behavior. [...] Steering positively with the desperate vec

...

Stephen Martin's Shortform

3

Stephen Martin's Shortform

3