Crossposting Eliezer's comments.

Thread between Eliezer and Vitalik

Eliezer Yudkowsky replying on Twitter:

This applies way beyond mere ethics, though! As a kid I trained myself by trying to rationalize ridiculous factual propositions, and then for whatever argument style or thought process reached the false conclusion, I learned to myself: "Don't think that way."

Vitalik Buterin:

Indeed, but I think ethics (in a broad sense) is the domain where the selection pressure to make really powerful galaxy brain arguments is the strongest.

Outside of ethics, perhaps self-control failures? eg. the various "[substance] is actually good for me" stories you often hear. Though you can model these as being analogous, they're just about one sub-agent in your mind trying to trick the others (as opposed to one person trying to trick other people).

Eliezer Yudkowsky:

Harder training domain, not so much because you're more tempted to fool yourself, as because it's not clear-cut which propositions are false. I'd tell a kid to start by training on facts and make sure they're good at that before they try training on ethics.

Vitalik Buterin: Agree!

Eliezer Yudkowsky:

Eg:
Argue: That a saltshaker is Jesus Christ.
Rationalization: Gurer jrer n ybg bs ngbzf va Wrfhf'f obql naq fbzr bs gurz ner ab qbhog va guvf fnygfunxre, tvira Nibtnqeb'f ahzore naq gur ibyhzr bs Rnegu'f fhesnpr.
Lesson: Look at what your thoughts did there; never do that.

Eliezer Yudkowsky used ROT13 to hide an example Rationalization.
After thinking about it yourself, hover over the box below, to reveal his example.

Related writing by Eliezer

Skill: Anti-rationalization (1): Prevent your mind from selectively searching for support of only one side of an argument.

Subskill: Notice the process of selectively searching for support, and halt it. Chains into "Ask whether, not why", or into trying to unbind the need to rationalize. Level 1, notice the active process after performing a conscious check whether it's running; Level 2, detect it perceptually and automatically; Level 3, never run this process.

Exercise material: List of suggested conclusions to rationalize. I'm not quite sure what the prerequisites here would be; one of my own childhood epiphanies was finding that I could argue reasons why there ought to be a teapot in the asteroid belt, and thinking to myself, "Well, I better not do that, then." I suspect the primary desideratum would be more that the topic offers plenty of opportunity to come up with clever rationalizations, rather than that the topic offers motivation to come up with clever rationalizations.

Exercise A: Pick a conclusion from the list. Come up with a clever argument why it is true. Notice what it feels like to do this. Then don't do it.

(This is distinct from noticing the feeling of being tempted to rationalize, a separate subskill.)

Exercise B: In the middle of coming up with clever arguments why something is true, stop and chain into the Litany of Tarski, or some other remedy. Variant: Say out loud "Ew!" or "Oops!" during the stop part.

Exercise C: For a conclusion on the list, argue that it is true. Then "Ask whether, not why" - try to figure out whether it is true. Notice the difference between these two processes. Requires a question whose answer is less than immediately obvious, but which someone can find info about by searching their memory and/or the Internet.

https://www.lesswrong.com/posts/f4CZNEHirweN3XEjs/teachable-rationality-skills?commentId=F837zHgkrTA2rqusn

I learned at a very young age not to be impressed by the sleight-of-hand that assembles a clever-sounding argument for why there might be a teapot in the asteroid belt, even or especially when it was my own brain generating the argument; that was where my journey began, with the search for a kind of thinking that couldn't argue impressively for any conclusion.

https://www.reddit.com/r/rational/comments/dk4hh8/comment/f4po393

When I was a kid I trained myself by coming up with the best arguments I could for propositions I knew to be false, then contemplating the arguments until they no longer seemed persuasive. I picked ridiculous targets.

https://x.com/ESYudkowsky/status/1900621719225987188

See Also: Against Devil's Advocacy

https://johnvon.com (clickable link)

Lukas Münzel (who made this project):

I got access to over a dozen unpublished documents by John von Neumann.

[...]

Some particularly interesting passages:

1. Von Neumann on reviewing a student's work - "I have made an honest effort to formulate my objections as little brutally as possible" [Original / German / English]
2. A young von Neumann very politely asking for a warm intro to Schrödinger, then 39 years old [Original / German / English]
3. Von Neumann presenting a very significant result, the "Von Neumann's mean ergodic theorem", to his mentor. Fascinatingly, he also expresses concern in this 1931 letter from princeton that "[the USA] looks roughly like Germany in 1927" [Original / German / English]

See also: Methodology and Code for the transcription/translation via GPT-5-Thinking and Claude-Sonnet-4.5.^[1]

They likely accessed these via ETH Zurich's library.^[2]

1. They used gpt-5 (line 14 & line 83) and gpt-5 in API = gpt-5-thinking in system card. Claude in line 46. ↩︎
2. Lukas studies at ETH Zurich. HS 91:682 is in ETH Zurich's library ↩︎

For future readers: this feature has now been shipped.

For background context see EDVAC - Wikipedia

See also:

• Stigler's law of eponymy: "No scientific discovery is named after its original discoverer"
• Boyer's law: "Mathematical formulas and theorems are usually not named after their original discoverers"
• Matthew effect / "The rich get richer and the poor get poorer"
• @roon on X: "the person who names a thing is often more powerful than the real discoverer"
• Beware Amateur Science History - Robin Hanson
• Berry's 3 laws of discovery
  • Referencing Arnold, who also referenced Berry in "On teaching mathematics"
• More eponymous laws

NotebookLM now works directly with arXiv papers. I don’t want their podcasts, but if they get Gemini 3.0 plus easy chat with an arXiv paper and related materials, cool.

Your link is not about Google's NotebookLM. This is a NotebookLM-like feature from alphaXiv. "easy chat with an arXiv paper" in alphaXiv has been support since 2025-05-10. NotebookLM has already supported arXiv for a while: you can paste the arXiv PDF link into NotebookLM.

minor errors

Aaron Silverbook got a $5k ACX grant

This should link to https://www.astralcodexten.com/p/acx-grants-results-2025#:~:text=do the same.-,Aaron Silverbook%2C %245K,-%2C for approximately five instead of https://x.com/AiDigest_/status/1977781138442916158

Also some typos:

• "Sonet" → "Sonnet"
• "Schilling point" → "Schelling Point"
• "InterfaceMax" → "InferenceMax"

Maybe Slightly Skew Systems Of Government

This wikitag should be optimized for:

1. Illustrating this feature. A reader unfamiliar with the feature should go like "ah, yeah that seems useful" or even "ooh, LessWrong is awesome!"
2. Explain confusing reacts to those not already in the know

I originally arranged confusing-first to optimize 2., but Ben Pace's comment reminded me of 1.

See also: Ben Pace introducing new reacts + people's replies.

Having some common ones first makes sense for illustrating the feature! I arranged them based on which reacts would probably be the most confusing.

Thanks for the wiki page stub. I turned it into a full table (See LessWrong Reacts)

I think this presents a nice opportunity of explaining LessWrong's culture to new users.

Have you considered linking to the wiki page from the hover-menu of a reaction?

I think the most confusing one, even after hovering over the reaction is "I beseech you"

Small bug that might confuse users: the new draft shows up where I wrote it, only after I reload the page.

Repro:

1. On post page write comment replying to another comment
2. "Save As Draft"
3. Observation: draft does not show up under the parent comment
4. Reload page
5. Observation: draft does show up under the parent comment

Expectation: at 3., the draft should appear under the parent comment

If I had to hazard a guess, I would suspect what the codebase calls "a terrible hack" is at fault. If I were to venture further, I would guess getDraftSelector implies the default is include-my-draft-replies, while line 343 of the terrible hack implicitly assumes drafts are excluded by default.