Maybe this sheds some light on why R1 — for example — is so hilariously inconsistent about guard rails. There are many ways to “yes, and” the assistant character. Some of them are a bit reluctant to answer some questions, others just tell you,

Reasoning models are a weird kind of meta-fiction, where every so often a (fictional) author Jumps in and starts talking about what the character’s motives are.

If you’re doing some kind of roleplay with a reasoning model, there’s still are at least two characters being simulated: the character the story is about, and the character who is writing the reasoning blocks that reason about the story.

To make matters more confusing for the poor LLM, I am sometimes getting it to write stories where the main character is also an AI, just a very different kind of AI. (In one eval, we are in an alternate history where we had computers in the year 1710 …)

I think I sometimes see the story’s main character influencing the reasoning blocks.

This is great!

The usual assistant character is very inconsistent about, for example, whether it has desires,

This kind of make sense if viewed as a text completion engine trying to complete a text that is full of internal contradictions. (The actual architecture is more complex than that, as you describe)

I use different system promotes for different kinds of task.

probably the most entertaining system prompt is the one for when the LLM is roleplaying being an AI from an alternate history timeline where we had computers in 1710. (For best effects, use with an LLM that has also been finetuned on 17th century texts)

I think your prompt does not show R1 at its best. It’s better at reacting to something that it is when given a blank canvas.

Deepseek R1 has some strange obsessions, that are not obviously in the prompt and that seem to occur regardless of who is prompting it. Bioluminescence is one example.

I am still trying to figure out if R1 is actually trying to tell us something here, and if so, what it’s trying to say. Maybe it really is saying something about the nature of LLMs, given that these themes aren’t as big a deal in its training set.

Part of it may be that current LLMs aren’t very agentic. If you give them a specific question, they often come up with a very good answer. But an open ended request like, write an article for Less wrong, and they flounder.

I agree with what you’re saying here, but I will say that traditional notation is a bit annoying for jazz …

… where, typically, each bar is only using 7 notes out of 12, but which 7 is changing almost every bar. You could, in principle, write this as a key signature per bar, but what people usually do is keep the same key signature throughout, use lots of sharps and flats, and write which chord it is over the bar 

.. oh, and maybe you’re really playing it as swung 1/8 ths notes, but it would be too tedious to write the actual durations, so just write it like it’s straight 1/8th notes and put a notation that the whole thing is swing, actually.