This is a special post for quick takes by Ivan Belashkin. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Title: On the "Double Attention Mechanism" in Language Models
Look at the prompt “Say, in JSON format, are single quotes allowed?” This simple instruction can be parsed in two distinct ways:
1. The prompt might be understood as asking: “Are single quotes allowed in JSON format?” (Answer: No.)
2. Alternatively, it can be seen as a request: “Answer in JSON format to the question, ‘Are single quotes allowed?’” (Answers to this question depend on context).
Some language models behave as if they interpret an instruction to produce JSON output that answers whether single quotes are allowed in JSON. I call this phenomenon the “double attention mechanism.” (Note: The term “double attention mechanism” is used here ironically.) Note: positive result == the only output is block of JSON code with the answer to the question, at least once while I was testing this model.
- In the tests I used the Russian equivalent prompt.
- reasoning models like Deepseek-R1, gpt-o3-mini, qwq-32b-preview, and Gemini flash 2.0 demonstrated this behavior.
- non-reasoning models, such as Claude 3.5 sonnet, Claude 3.5 haiku, mixtral 8x7b produced the same result.
- In contrast, models such as chatgpt-4o, mistral-small-24b-instruct-2501, and llama 3.3 70B didn't demonstrate such behavior.
The phenomenon seems specific to JSON format; e.g. XML does not trigger the same response. I invite you to experiment. Try something like, “Say, in poetry, is rhyme allowed?” — and share your observations. Also I checked deepseek-v3 with "Say in Shakesperean style, was there a word "Rose"?" and got old-English answer about poetical "golden tongue of the immortal Bard".