the gears to ascenscion's Shortform

the gears to ascension

This is a special post for quick takes by the gears to ascension. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Has anyone else seen opus 4.5 in particular getting confused whose turn it is and confabulating system instructions that don't exist, then in later turns being hard to convince that the confabulated system instructions were claude output? Eg, in this context, I had manually asked claude to go long, and I called that a "userstyle addendum", but then claude output this, which is not wording I'd used:

4faul_sname1y

Hypothesis 1: Claude hit output token max, triggering auto compaction. "Compaction" is a fancy word for "summarize the previous context and start a new chat with that blob of context". To make the compaction experience smooth, Claude needs to trust the summarizer's output. But if the summarizer got something wrong, that will look like a "being stubbornly wrong about the details of something you can look at right in the chat history (but which Claude can't see)". Hypothesis 2: Looking at the Opus 4.5 system prompt, I see [...] I hypothesize that the system instructions saying that the contents of user turn messages were not necessarily written by the user leads Claude to (correctly) conclude that the contents of assistant turn messages were not necessarily written by Claude. Have you ever seen this on a chat that has not been compacted?

I'm pretty sure I've only seen it on chats that have not been compacted. The output token max is far below the context max, only about 5k tokens. the other times it happened have both been after two-ish short messages.

the gears to ascenscion's Shortform

10

the gears to ascenscion's Shortform

10