Lao Mein's Shortform

Lao Mein

This is a special post for quick takes by Lao Mein. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I've noticed a consistent pattern in LLMs being able to recall articles on LessWrong that have had near-zero interactions (I think there were 2 comments on my post mentioned here). The behavior is seen across every model I've tested, including non-search capable DeepSeek versions. LW is ~~massively~~^[1] probably oversampled in LLM training text. If you've written something distinctive here, even if no one has commented or even voted on your post, LLMs ~~probably~~ possibly know about it, and ~~possibly~~ may know you.

edit: I experimented a lot more, and I think "interesting enough for Claude to remember the title/topic/author of the post" has a threshold of ~50 karma. It remembers people a lot better than specific posts though, but honestly a single interesting post and a comment history is enough for it to recall roughly what posts you made in a particular year. It managed to recall every significant post I made in 2024 (other than my powerplant one), which is impressive.

The GPT-4o stuff was because I was having comment chains with Gwern about the GPT-4o tokenizer. No post on DS tokens though, although I remember making a few comments.

^{^}
retracted, got tricked by Gemini hallucination.

I tried this, and Claude wasn't familiar with my article, but seemed to be strangely good at guessing its contents:

https://claude.ai/share/843f7a21-b7f3-4ce6-986e-8f350450fff5

Gemini completely fails to recall anything about it and its guesses are random:

https://share.gemini.google/EHqtVcVyit6G

I don't have paid ChatGPT but the free one at least failed too, and kept trying to cheat and search the web anyway, even given 4 instructions not to in the prompt.

2the gears to ascension2mo

I figured this was just that they're overtrained to memorization. To know if this is the case you'd need to check articles that are similarly unique but much less likely to be overrepresented in training data, and compare memorization between them. I expect they're just training until memorization for most models, because actually there's no good reason to not do that as long as you also get generalization - a solomonoff inductor selects only the programs which perfectly memorize, which means so long as memorization doesn't come with effective program length increase or degrade the representations you're training to get (both big ifs!), you just want to always memorize.

2Lao Mein2mo

Memorization? It might be memorizing, but it's not memorizing the original text. It can't do verbatum recall. When it does attempt verbatum recall, it actually produces a summary. It has trouble delineating between SolidGoldMagikarp III: Glitch token archaeology to my catalog post. Now, we actually write in very different styles, so... yeah. It's probably being trained on Claude summaries of ML text. Maybe also to memorization. Should be easy to test.