Brendan Long's Shortform

Brendan Long

This is a special post for quick takes by Brendan Long. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

Fable^[1] can one-shot "make Pangram think this text is 100% human".

Before (100% AI): https://www.pangram.com/history/5388833f-b611-4605-bbcb-f34dd7ee4294

After (100% human): https://www.pangram.com/history/9c247abc-427a-4916-badd-bb42e6cb4d85

Both texts were written by Fable with the goal of convincing Twitter's broken support bot to unban me (full conversation log). For the initial one it wasn't told to evade Pangram.

The prompt to de-Pangram it was:

Hm I think the human version is best, although I worry they will also run these through an AI detector (Pangram confirms 100% AI written). Can you write to write this in a way that doesn't sound AI?

^{^}
I'm actually kind of surprised this didn't get a refusal, but I'm not sure if claude.ai tells you if Opus fallback is happening or not.

4Coagulopath3mo

I think detection at ~150 words is pretty unreliable. Even lowly GPT 3.5 (now nearly four old) partially evades detection with one prompt ("Rewrite in a way that doesn't sound AI"), and with a second pass achieves 100% human authorship. It doesn't help that Pangram has never seen Fable's output before. The latest version, 3.3, was trained in May. So Fable (and GPT-5.6, and Kimi K-3, and...) will be a bit OOD until the next release. Fable's logic is interesting. It surely knows about Pangram but its "humanizing" seems mostly designed to fool human readers: a statistical model doesn't care much about whether you have "tidy evidence lists" and a tone of "dry frustration". The rewritten pass still has a number of AI tells, like tricolons ("No automation, no spam, nothing coordinated"), forced plot twists ("Except it wasn't.") it's not x, it's y patterns ("I'm not really arguing the classification, I'm asking someone to actually apply the reversal"), and hedging. It mixes unlike conversational registers: would a person who writes the stiffly formal "For whatever it's worth on the authenticity question" also write the casual "stuff I wrote myself"? I guess Fable did the wrong things and evaded detection by accident? Sadly, Pangram isn't magic and folds pretty easily to tricks that the average LWer could think of in a few minutes. You could probably make a Claude skill that gets to 100% at this point.

2Brendan Long3mo

I think Claude is partially right about the evidence lists. I think Pangram's evidence mode (which I couldn't find today) has flagged bullet points as suspicious for me in the past.

Pangram made it very clear that the supporting evidence is a different model than the one that generates the probabilities. I would guess they removed it because people still keep getting confused.