LESSWRONG
LW

Frontpage

7

[ Question ]

Are LLMs being trained using LessWrong text?

by Cedar
2nd Jul 2025
1 min read
A
3
4

7

Frontpage

7

Are LLMs being trained using LessWrong text?
8avturchin
7Gordon Seidoh Worley
1Cedar
3Viliam
New Answer
New Comment

3 Answers sorted by
top scoring

avturchin

Jul 02, 2025

80

Yes, they can generate a list of comments to a post, putting correct names of prominent LessWrongers and typical styles and topics for each commenter. 

Add Comment

Gordon Seidoh Worley

Jul 02, 2025

72

Experimentally, Claude knows details about things I specifically wrote on Less Wrong without doing a web search, as well as other Less Wrong content. I'm fairly confident Less Wrong posts are in its training set and not gotten from mirrors other places.

Add Comment

Cedar

Jul 02, 2025

10

LessWrong scrape dataset on Hugging face, by NousResearch 

https://huggingface.co/datasets/LDJnr/LessWrong-Amplify-Instruct

Add Comment
1 comment, sorted by
top scoring
Click to highlight new comments since: Today at 7:13 PM
[-]Viliam12d35

Potentially good news is that we might contribute to raising the LLM sanity waterline?

Makes me wonder, when LLMs are trained on texts not just from LW but also from Reddit, is the karma information included? That is, is upvoted content somehow considered more important than downvoted, or is it treated all the same way?

If it is all the same, maybe the datasets could be improved by removing negative-karma content?

Reply
Moderation Log
Curated and popular this week
A
3
1

I wonder if there's a clear evidence that LessWrong text has been included in LLM training.

Claude seems generally aware of LessWrong, but it's difficult to distinguish between "this model has been trained on text that mentions LessWrong" and "this model has been trained on text from LessWrong"

Related discussion here, about preventing inclusion: https://www.lesswrong.com/posts/SGDjWC9NWxXWmkL86/keeping-content-out-of-llm-training-datasets?utm_source=perplexity