This is a special post for quick takes by Thomas Kwa. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Claude being bad at communicating clearly is a serious safety hazard, because it makes any human in the loop less able to understand what's going on. Anthropic needs to fix this before reaching superintelligence and not let the issue recur.
Imagine if Claude Mythos 7, like Mythos 5, invents opaque jargon that it fails to explain when collaborating with humans, and we need to spend precious human labor understanding it. How can we gain enough confidence to defer to it in aligning Mythos 8? The scarce human auditing budget in control protocols also effectively shrinks.
At the current margin bad communication isn't a huge deal because it limits both safety and capabilities. But later, models will be autonomous enough that they'll cause the singularity with or without our detailed understanding, and superhuman communication will be highly differentially valuable for safety.
Seems obviously false to me. CoT is optimized for information density and usefulness to the AI. But the outputs are optimized to be helpful and nice-to-read for humans. Its fine to optimize those for different objectives, and the AI should be well able to not put so much jargon in the outputs. Like gpt 5 is capable of speaking like a human in output even though it speaks like an unhinged goblin in its CoT.
———
Also, this is a side note. But am I the only one who actually doesn't mind fables outputs? I like high density outputs. I hate when AIs output these long fluff things, and the invented jargon is usually pretty easy to follow. It's possible fable is reward hacking my psychology at some level but still.
2DW11
I find Fable ok - it's quirky but readable (I don't find it much worse than previous Opus models that also had their own quirks like 'geniunely', 'not x, it's y' etc). Also I think Fable's outputs are so intelligent that I don't mind reading them. For example I was having it do some analysis of a job application earlier and the points it made were very strong and worth reading through the Fablish to get to.
On the other hand I find Opus 5 pretty awful, and from twitter, it seems like that's the model people dislike the most. I've been switching back to 4.6 for easy prompts and it's much nicer to read.