LESSWRONG
The Best of LessWrong
LW

1934

The Best of LessWrong

When posts turn more than a year old, the LessWrong community reviews and votes on how well they have stood the test of time. These are the posts that have ranked the highest for all years since 2018 (when our annual tradition of choosing the least wrong of LessWrong began).

For the years 2018, 2019 and 2020 we also published physical books with the results of our annual vote, which you can buy and learn more about here.
+

Rationality

Eliezer Yudkowsky
Local Validity as a Key to Sanity and Civilization
Buck
"Other people are wrong" vs "I am right"
Mark Xu
Strong Evidence is Common
TsviBT
Please don't throw your mind away
Raemon
Noticing Frame Differences
johnswentworth
You Are Not Measuring What You Think You Are Measuring
johnswentworth
Gears-Level Models are Capital Investments
Hazard
How to Ignore Your Emotions (while also thinking you're awesome at emotions)
Scott Garrabrant
Yes Requires the Possibility of No
Ben Pace
A Sketch of Good Communication
Eliezer Yudkowsky
Meta-Honesty: Firming Up Honesty Around Its Edge-Cases
Duncan Sabien (Inactive)
Lies, Damn Lies, and Fabricated Options
Scott Alexander
Trapped Priors As A Basic Problem Of Rationality
Duncan Sabien (Inactive)
Split and Commit
Duncan Sabien (Inactive)
CFAR Participant Handbook now available to all
johnswentworth
What Are You Tracking In Your Head?
Mark Xu
The First Sample Gives the Most Information
Duncan Sabien (Inactive)
Shoulder Advisors 101
Scott Alexander
Varieties Of Argumentative Experience
Eliezer Yudkowsky
Toolbox-thinking and Law-thinking
alkjash
Babble
Zack_M_Davis
Feature Selection
abramdemski
Mistakes with Conservation of Expected Evidence
Kaj_Sotala
The Felt Sense: What, Why and How
Duncan Sabien (Inactive)
Cup-Stacking Skills (or, Reflexive Involuntary Mental Motions)
Ben Pace
The Costly Coordination Mechanism of Common Knowledge
Jacob Falkovich
Seeing the Smoke
Duncan Sabien (Inactive)
Basics of Rationalist Discourse
alkjash
Prune
johnswentworth
Gears vs Behavior
Elizabeth
Epistemic Legibility
Daniel Kokotajlo
Taboo "Outside View"
Duncan Sabien (Inactive)
Sazen
AnnaSalamon
Reality-Revealing and Reality-Masking Puzzles
Eliezer Yudkowsky
ProjectLawful.com: Eliezer's latest story, past 1M words
Eliezer Yudkowsky
Self-Integrity and the Drowning Child
Jacob Falkovich
The Treacherous Path to Rationality
Scott Garrabrant
Tyranny of the Epistemic Majority
alkjash
More Babble
abramdemski
Most Prisoner's Dilemmas are Stag Hunts; Most Stag Hunts are Schelling Problems
Raemon
Being a Robust Agent
Zack_M_Davis
Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists
Benquo
Reason isn't magic
habryka
Integrity and accountability are core parts of rationality
Raemon
The Schelling Choice is "Rabbit", not "Stag"
Diffractor
Threat-Resistant Bargaining Megapost: Introducing the ROSE Value
Raemon
Propagating Facts into Aesthetics
johnswentworth
Simulacrum 3 As Stag-Hunt Strategy
LoganStrohl
Catching the Spark
Jacob Falkovich
Is Rationalist Self-Improvement Real?
Benquo
Excerpts from a larger discussion about simulacra
Zvi
Simulacra Levels and their Interactions
abramdemski
Radical Probabilism
sarahconstantin
Naming the Nameless
AnnaSalamon
Comment reply: my low-quality thoughts on why CFAR didn't get farther with a "real/efficacious art of rationality"
Eric Raymond
Rationalism before the Sequences
Owain_Evans
The Rationalists of the 1950s (and before) also called themselves “Rationalists”
Raemon
Feedbackloop-first Rationality
LoganStrohl
Fucking Goddamn Basics of Rationalist Discourse
Raemon
Tuning your Cognitive Strategies
johnswentworth
Lessons On How To Get Things Right On The First Try
+

Optimization

So8res
Focus on the places where you feel shocked everyone's dropping the ball
Jameson Quinn
A voting theory primer for rationalists
sarahconstantin
The Pavlov Strategy
Zvi
Prediction Markets: When Do They Work?
johnswentworth
Being the (Pareto) Best in the World
alkjash
Is Success the Enemy of Freedom? (Full)
johnswentworth
Coordination as a Scarce Resource
AnnaSalamon
What should you change in response to an "emergency"? And AI risk
jasoncrawford
How factories were made safe
HoldenKarnofsky
All Possible Views About Humanity's Future Are Wild
jasoncrawford
Why has nuclear power been a flop?
Zvi
Simple Rules of Law
Scott Alexander
The Tails Coming Apart As Metaphor For Life
Zvi
Asymmetric Justice
Jeffrey Ladish
Nuclear war is unlikely to cause human extinction
Elizabeth
Power Buys You Distance From The Crime
Eliezer Yudkowsky
Is Clickbait Destroying Our General Intelligence?
Spiracular
Bioinfohazards
Zvi
Moloch Hasn’t Won
Zvi
Motive Ambiguity
Benquo
Can crimes be discussed literally?
johnswentworth
When Money Is Abundant, Knowledge Is The Real Wealth
GeneSmith
Significantly Enhancing Adult Intelligence With Gene Editing May Be Possible
HoldenKarnofsky
This Can't Go On
Said Achmiz
The Real Rules Have No Exceptions
Lars Doucet
Lars Doucet's Georgism series on Astral Codex Ten
johnswentworth
Working With Monsters
jasoncrawford
Why haven't we celebrated any major achievements lately?
abramdemski
The Credit Assignment Problem
Martin Sustrik
Inadequate Equilibria vs. Governance of the Commons
Scott Alexander
Studies On Slack
KatjaGrace
Discontinuous progress in history: an update
Scott Alexander
Rule Thinkers In, Not Out
Raemon
The Amish, and Strategic Norms around Technology
Zvi
Blackmail
HoldenKarnofsky
Nonprofit Boards are Weird
Wei Dai
Beyond Astronomical Waste
johnswentworth
Making Vaccine
jefftk
Make more land
jenn
Things I Learned by Spending Five Thousand Hours In Non-EA Charities
Richard_Ngo
The ants and the grasshopper
So8res
Enemies vs Malefactors
Elizabeth
Change my mind: Veganism entails trade-offs, and health is one of the axes
+

World

Kaj_Sotala
Book summary: Unlocking the Emotional Brain
Ben
The Redaction Machine
Samo Burja
On the Loss and Preservation of Knowledge
Alex_Altair
Introduction to abstract entropy
Martin Sustrik
Swiss Political System: More than You ever Wanted to Know (I.)
johnswentworth
Interfaces as a Scarce Resource
eukaryote
There’s no such thing as a tree (phylogenetically)
Scott Alexander
Is Science Slowing Down?
Martin Sustrik
Anti-social Punishment
johnswentworth
Transportation as a Constraint
Martin Sustrik
Research: Rescuers during the Holocaust
GeneSmith
Toni Kurz and the Insanity of Climbing Mountains
johnswentworth
Book Review: Design Principles of Biological Circuits
Elizabeth
Literature Review: Distributed Teams
Valentine
The Intelligent Social Web
eukaryote
Spaghetti Towers
Eli Tyre
Historical mathematicians exhibit a birth order effect too
johnswentworth
What Money Cannot Buy
Bird Concept
Unconscious Economics
Scott Alexander
Book Review: The Secret Of Our Success
johnswentworth
Specializing in Problems We Don't Understand
KatjaGrace
Why did everything take so long?
Ruby
[Answer] Why wasn't science invented in China?
Scott Alexander
Mental Mountains
L Rudolf L
A Disneyland Without Children
johnswentworth
Evolution of Modularity
johnswentworth
Science in a High-Dimensional World
Kaj_Sotala
My attempt to explain Looking, insight meditation, and enlightenment in non-mysterious terms
Kaj_Sotala
Building up to an Internal Family Systems model
Steven Byrnes
My computational framework for the brain
Natália
Counter-theses on Sleep
abramdemski
What makes people intellectually active?
Bucky
Birth order effect found in Nobel Laureates in Physics
zhukeepa
How uniform is the neocortex?
JackH
Anti-Aging: State of the Art
Vaniver
Steelmanning Divination
KatjaGrace
Elephant seal 2
Zvi
Book Review: Going Infinite
Rafael Harth
Why it's so hard to talk about Consciousness
Duncan Sabien (Inactive)
Social Dark Matter
Eric Neyman
How much do you believe your results?
Malmesbury
The Talk: a brief explanation of sexual dimorphism
moridinamael
The Parable of the King and the Random Process
Henrik Karlsson
Cultivating a state of mind where new ideas are born
+

Practical

alkjash
Pain is not the unit of Effort
benkuhn
Staring into the abyss as a core life skill
Unreal
Rest Days vs Recovery Days
Duncan Sabien (Inactive)
In My Culture
juliawise
Notes from "Don't Shoot the Dog"
Elizabeth
Luck based medicine: my resentful story of becoming a medical miracle
johnswentworth
How To Write Quickly While Maintaining Epistemic Rigor
Duncan Sabien (Inactive)
Ruling Out Everything Else
johnswentworth
Paper-Reading for Gears
Elizabeth
Butterfly Ideas
Eliezer Yudkowsky
Your Cheerful Price
benkuhn
To listen well, get curious
Wei Dai
Forum participation as a research strategy
HoldenKarnofsky
Useful Vices for Wicked Problems
pjeby
The Curse Of The Counterfactual
Darmani
Leaky Delegation: You are not a Commodity
Adam Zerner
Losing the root for the tree
chanamessinger
The Onion Test for Personal and Institutional Honesty
Raemon
You Get About Five Words
HoldenKarnofsky
Learning By Writing
GeneSmith
How to have Polygenically Screened Children
AnnaSalamon
“PR” is corrosive; “reputation” is not.
Ruby
Do you fear the rock or the hard place?
johnswentworth
Slack Has Positive Externalities For Groups
Raemon
Limerence Messes Up Your Rationality Real Bad, Yo
mingyuan
Cryonics signup guide #1: Overview
catherio
microCOVID.org: A tool to estimate COVID risk from common activities
Valentine
Noticing the Taste of Lotus
orthonormal
The Loudest Alarm Is Probably False
Raemon
"Can you keep this confidential? How do you know?"
mingyuan
Guide to rationalist interior decorating
Screwtape
Loudly Give Up, Don't Quietly Fade
+

AI Strategy

paulfchristiano
Arguments about fast takeoff
Eliezer Yudkowsky
Six Dimensions of Operational Adequacy in AGI Projects
Ajeya Cotra
Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover
paulfchristiano
What failure looks like
Daniel Kokotajlo
What 2026 looks like
gwern
It Looks Like You're Trying To Take Over The World
Daniel Kokotajlo
Cortés, Pizarro, and Afonso as Precedents for Takeover
Daniel Kokotajlo
The date of AI Takeover is not the day the AI takes over
Andrew_Critch
What Multipolar Failure Looks Like, and Robust Agent-Agnostic Processes (RAAPs)
paulfchristiano
Another (outer) alignment failure story
Ajeya Cotra
Draft report on AI timelines
Eliezer Yudkowsky
Biology-Inspired AGI Timelines: The Trick That Never Works
Daniel Kokotajlo
Fun with +12 OOMs of Compute
Wei Dai
AI Safety "Success Stories"
Eliezer Yudkowsky
Pausing AI Developments Isn't Enough. We Need to Shut it All Down
HoldenKarnofsky
Reply to Eliezer on Biological Anchors
Richard_Ngo
AGI safety from first principles: Introduction
johnswentworth
The Plan
Rohin Shah
Reframing Superintelligence: Comprehensive AI Services as General Intelligence
lc
What an actually pessimistic containment strategy looks like
Eliezer Yudkowsky
MIRI announces new "Death With Dignity" strategy
KatjaGrace
Counterarguments to the basic AI x-risk case
Adam Scholl
Safetywashing
habryka
AI Timelines
evhub
Chris Olah’s views on AGI safety
So8res
Comments on Carlsmith's “Is power-seeking AI an existential risk?”
nostalgebraist
human psycholinguists: a critical appraisal
nostalgebraist
larger language models may disappoint you [or, an eternally unfinished draft]
Orpheus16
Speaking to Congressional staffers about AI risk
Tom Davidson
What a compute-centric framework says about AI takeoff speeds
abramdemski
The Parable of Predict-O-Matic
KatjaGrace
Let’s think about slowing down AI
Daniel Kokotajlo
Against GDP as a metric for timelines and takeoff speeds
Joe Carlsmith
Predictable updating about AI risk
Raemon
"Carefully Bootstrapped Alignment" is organizationally hard
KatjaGrace
We don’t trade with ants
+

Technical AI Safety

paulfchristiano
Where I agree and disagree with Eliezer
Eliezer Yudkowsky
Ngo and Yudkowsky on alignment difficulty
Andrew_Critch
Some AI research areas and their relevance to existential safety
1a3orn
EfficientZero: How It Works
elspood
Security Mindset: Lessons from 20+ years of Software Security Failures Relevant to AGI Alignment
So8res
Decision theory does not imply that we get to have nice things
Vika
Specification gaming examples in AI
Rafael Harth
Inner Alignment: Explain like I'm 12 Edition
evhub
An overview of 11 proposals for building safe advanced AI
TurnTrout
Reward is not the optimization target
johnswentworth
Worlds Where Iterative Design Fails
johnswentworth
Alignment By Default
johnswentworth
How To Go From Interpretability To Alignment: Just Retarget The Search
Alex Flint
Search versus design
abramdemski
Selection vs Control
Buck
AI Control: Improving Safety Despite Intentional Subversion
Eliezer Yudkowsky
The Rocket Alignment Problem
Eliezer Yudkowsky
AGI Ruin: A List of Lethalities
Mark Xu
The Solomonoff Prior is Malign
paulfchristiano
My research methodology
TurnTrout
Reframing Impact
Scott Garrabrant
Robustness to Scale
paulfchristiano
Inaccessible information
TurnTrout
Seeking Power is Often Convergently Instrumental in MDPs
So8res
A central AI alignment problem: capabilities generalization, and the sharp left turn
evhub
Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research
paulfchristiano
The strategy-stealing assumption
So8res
On how various plans miss the hard bits of the alignment challenge
abramdemski
Alignment Research Field Guide
johnswentworth
The Pointers Problem: Human Values Are A Function Of Humans' Latent Variables
Buck
Language models seem to be much better than humans at next-token prediction
abramdemski
An Untrollable Mathematician Illustrated
abramdemski
An Orthodox Case Against Utility Functions
Veedrac
Optimality is the tiger, and agents are its teeth
Sam Ringer
Models Don't "Get Reward"
Alex Flint
The ground of optimization
johnswentworth
Selection Theorems: A Program For Understanding Agents
Rohin Shah
Coherence arguments do not entail goal-directed behavior
abramdemski
Embedded Agents
evhub
Risks from Learned Optimization: Introduction
nostalgebraist
chinchilla's wild implications
johnswentworth
Why Agent Foundations? An Overly Abstract Explanation
zhukeepa
Paul's research agenda FAQ
Eliezer Yudkowsky
Coherent decisions imply consistent utilities
paulfchristiano
Open question: are minimal circuits daemon-free?
evhub
Gradient hacking
janus
Simulators
LawrenceC
Causal Scrubbing: a method for rigorously testing interpretability hypotheses [Redwood Research]
TurnTrout
Humans provide an untapped wealth of evidence about alignment
Neel Nanda
A Mechanistic Interpretability Analysis of Grokking
Collin
How "Discovering Latent Knowledge in Language Models Without Supervision" Fits Into a Broader Alignment Scheme
evhub
Understanding “Deep Double Descent”
Quintin Pope
The shard theory of human values
TurnTrout
Inner and outer alignment decompose one hard problem into two extremely hard problems
Eliezer Yudkowsky
Challenges to Christiano’s capability amplification proposal
Scott Garrabrant
Finite Factored Sets
paulfchristiano
ARC's first technical report: Eliciting Latent Knowledge
Diffractor
Introduction To The Infra-Bayesianism Sequence
TurnTrout
Towards a New Impact Measure
LawrenceC
Natural Abstractions: Key Claims, Theorems, and Critiques
Zack_M_Davis
Alignment Implications of LLM Successes: a Debate in One Act
johnswentworth
Natural Latents: The Math
TurnTrout
Steering GPT-2-XL by adding an activation vector
Jessica Rumbelow
SolidGoldMagikarp (plus, prompt generation)
So8res
Deep Deceptiveness
Charbel-Raphaël
Davidad's Bold Plan for Alignment: An In-Depth Explanation
Charbel-Raphaël
Against Almost Every Theory of Impact of Interpretability
Joe Carlsmith
New report: "Scheming AIs: Will AIs fake alignment during training in order to get power?"
Eliezer Yudkowsky
GPTs are Predictors, not Imitators
peterbarnett
Labs should be explicit about why they are building AGI
HoldenKarnofsky
Discussion with Nate Soares on a key alignment difficulty
Jesse Hoogland
Neural networks generalize because of this one weird trick
paulfchristiano
My views on “doom”
technicalities
Shallow review of live agendas in alignment & safety
Vanessa Kosoy
The Learning-Theoretic Agenda: Status 2023
ryan_greenblatt
Improving the Welfare of AIs: A Nearcasted Proposal
201820192020202120222023All
RationalityWorldOptimizationAI StrategyTechnical AI SafetyPracticalAll
#2
“PR” is corrosive; “reputation” is not.

Anna Salamon argues that "PR" is a corrupt concept that can lead to harmful and confused actions, while safeguarding one's "reputation" or "honor" is generally fine. PR involves modeling what might upset people and avoiding it, while reputation is about adhering to fixed standards. 

by AnnaSalamon
#3
Your Cheerful Price

When negotiating prices for goods/services, Eliezer suggests asking for the other person's "Cheerful Price" - the price that would make them feel genuinely happy and enthusiastic about the transaction, rather than just grudgingly willing. This avoids social capital costs and ensures both parties feel good about the exchange.

by Eliezer Yudkowsky
#4
microCOVID.org: A tool to estimate COVID risk from common activities

How much COVID risk do you take when you go to the grocery store? When you see a friend outdoors? This calculator helps you estimate your risk from common activities in microcovids - units of 1-in-a-million chance of getting COVID. 

by catherio
#8
Pain is not the unit of Effort

Pain is often treated as a measure of effort. "No pain, no gain". But this attitude can be toxic and counterproductive. alkjash argues that if something hurts, you're probably doing it wrong, and that you're not trying your best if you're not happy. 

by alkjash
#8
Staring into the abyss as a core life skill

Ben observes that all favorite people are great at a skill he's labeled in my head as "staring into the abyss" – thinking reasonably about things that are uncomfortable to contemplate, like arguments against your religious beliefs, or in favor of breaking up with your partner. 

by benkuhn
#11
The Loudest Alarm Is Probably False

orthonormal reflects that different people experience different social fears. He guesses that the strongest fear for a person (an "alarm" in their head) is usually broken. So the people who are most selfless end up that way because uncalibrated fear they're being too selfish, the most loud are that because of the fear of not being heard, etc.

by orthonormal
#11
You Get About Five Words

When it comes to coordinating people around a goal, you don't get limitless communication bandwidth for conveying arbitrarily nuanced messages. Instead, the "amount of words" you get to communicate depends on how many people you're trying to coordinate. Once you have enough people....you don't get many words.

by Raemon
#11
How To Write Quickly While Maintaining Epistemic Rigor

There's a trick to writing quickly, while maintaining epistemic rigor: stop trying to justify your beliefs. Don't go looking for citations to back your claim. Instead, think about why you currently believe this thing, and try to accurately describe what led you to believe it.

by johnswentworth
#12
Luck based medicine: my resentful story of becoming a medical miracle

Elizabeth Van Nostrand spent literal decades seeing doctors about digestive problems that made her life miserable. She tried everything and nothing worked, until one day a doctor prescribed 5 different random supplements without thinking too hard about it and one of them miraculously cured her. This has led her to believe that sometimes you need to optimize for luck rather than scientific knowledge when it comes to medicine.

by Elizabeth
#12
How to have Polygenically Screened Children

Polygenic screening can increase your child's IQ by 2-8 points, decrease disease risk by up to 60%, and increase height by over 2 inches. Here's a detailed guide on how to maximize the benefits and minimize the costs of embryo selection. 

by GeneSmith
#14
Cryonics signup guide #1: Overview

A detailed guide on how to sign up for cryonics, for who have been vaguely meaning to sign up but felt intimidated. The first post has a simple action you can take to get you started.

by mingyuan
#16
Rest Days vs Recovery Days

A Recovery Day (or "slug day"), is where you're so tired you can only binge Netflix or stay in bed. A Rest Day is where you have enough energy to "follow your gut" with no obligations or pressure. Unreal argues that true rest days are important for avoiding burnout, and gives suggestions on how to implement them.

by Unreal
#25
Losing the root for the tree

A look at how we can get caught up in the details and lose sight of the bigger picture. By repeatedly asking "what are we really trying to accomplish here?", we can step back and refocus on what's truly important, whether in our careers, health, or life overall.

by Adam Zerner
#26
In My Culture

If you want to bring up a norm or expectation that's important to you, but not something you'd necessarily argue should be universal, an option is to preface it with the phrase "in my culture." In Duncan's experience, this helps navigate tricky situations by taking your own personal culture as object, and discussing how it is important to you without making demands of others.

by Duncan Sabien (Inactive)
#26
Slack Has Positive Externalities For Groups

When someone in a group has extra slack, it makes it easier for the whole group to coordinate, adapt, and take on opportunities. But individuals mostly don't reap the benefits, so aren't incentivized to maintain that extra slack. The post explores implications and possible solutions.

by johnswentworth
#26
Guide to rationalist interior decorating

Have you seen a Berkeley Rationalist house and thought "wow the lighting here is nice and it's so comfy" and vaguely wished your house had nice lighting and was comfy in that particular way? Well, this practical / anthropological guide should help.

by mingyuan
#28
Noticing the Taste of Lotus

You can learn to spot when something is hijacking your motivation, and take back control of your goals.

by Valentine
#28
Forum participation as a research strategy

In this post, I proclaim/endorse forum participation (aka commenting) as a productive research strategy that I've managed to stumble upon, and recommend it to others (at least to try). Note that this is different from saying that forum/blog posts are a good way for a research community to communicate. It's about individually doing better as researchers.

by Wei Dai
#29
Ruling Out Everything Else

When you're trying to communicate, a significant part of your job should be to proactively and explicitly rule out the most likely misunderstandings that your audience might jump to. Especially if you're saying something similar-to but distinct-from a common position that your audience will be familiar with.

by Duncan Sabien (Inactive)
#30
Leaky Delegation: You are not a Commodity

When considering buying something vs making/doing it yourself, there's a lot more to consider than just the price you'd pay and the opportunity cost of your time. Darmani covers several additional factors that can tip the scales in favor of DIY, including how outsourcing may result in a different product, how it can introduce principal-agent problems, and the value of learning. 

by Darmani
#31
"Can you keep this confidential? How do you know?"

People often ask "Can you keep this confidential?" without really checking if the person has the skills to do so. Raemon argues we need to be more careful about how we handle confidential informationm, and have explicit conversations about privacy practices. 

by Raemon
#33
Butterfly Ideas

Sometimes your brilliant, hyperanalytical friends can accidentally crush your fragile new ideas before they have a chance to develop. Elizabeth shares a strategy she uses to get them to chill out and vibe on new ideas for a bit before dissecting them. 

by Elizabeth
#35
Do you fear the rock or the hard place?

Heated, tense arguments can often be unproductive and unpleasant. Neither side feels heard, and they are often working desperately to defend something they feel is very important. Ruby explores this problem and some solutions.

by Ruby
#35
To listen well, get curious

You've probably heard the advice "to be a good listener, reflect back what people tell you." Ben Kuhn argues this is cargo cult advice that misses the point. The real key to good listening is intense curiosity about the details of the other person's situation. 

by benkuhn
#37
Useful Vices for Wicked Problems

When tackling difficult, open-ended research questions, it's easy to get stuck. In addition to vices like openmindedness and self-criticality, Holden recommends "vices" like laziness, impatience, hubris and self-preservation as antidotes. This post explores the techniques that have worked well for him.

by HoldenKarnofsky
#39
Loudly Give Up, Don't Quietly Fade

There's a supercharged version of the bystander effect where someone claims they'll do a task, but then quietly fails to follow through. This leaves others thinking the task is being handled when it's not. To prevent that, we should try to loudly announce when we're giving up on tasks we've taken on, rather than quietly fading away. And we should appreciate it when others do the same.

by Screwtape
#40
Paper-Reading for Gears

Most advice on reading scientific papers focuses on evaluating individual claims. But what if you want to build a deeper "gears-level" understanding of a system? John Wentworth offers advice on how to read papers to build such models, including focusing on boring details, reading broadly, and looking for mediating variables. 

by johnswentworth
#44
Notes from "Don't Shoot the Dog"

Karen Pryor's "Don't Shoot the Dog" applies behavioral psychology to training animals and people. Julia reads it as a parenting book, and shares key insights about reinforcing good behavior, avoiding accidentally rewarding bad behavior, and why clicker training works so well. 

by juliawise
#45
Learning By Writing

Holden shares his step-by-step process for forming opinions on a topic, developing and refining hypotheses, and ultimately arriving at a nuanced view - all while focusing on writing rather than just passively consuming information.

by HoldenKarnofsky
#46
The Curse Of The Counterfactual

Many of us are held back by mental patterns that compare reality to imaginary "shoulds". PJ Eby explains how to recognize this pattern and start to get free of it.

by pjeby
#46
Limerence Messes Up Your Rationality Real Bad, Yo

Limerence (aka "falling in love") wrecks havoc on your rationality. But it feels so good! 

What do?

by Raemon
#47
The Onion Test for Personal and Institutional Honesty

Do you pass the "onion test" for honesty? If people get to know you better over time, do they find out new things, but not be shocked by the *types* of information that were hidden? A framework for thinking about personal (and institutional) honesty. 

by chanamessinger
31Orpheus16
I read this post for the first time in 2022, and I came back to it at least twice.  What I found helpful * The proposed solution: I actually do come back to the “honor” frame sometimes. I have little Rob Bensinger and Anna Salamon shoulder models that remind me to act with integrity and honor. And these shoulder models are especially helpful when I’m noticing (unhelpful) concerns about social status. * A crisp and community-endorsed statement of the problem: It was nice to be like “oh yeah, this thing I’m experiencing is that thing that Anna Salamon calls PR.” And to be honest, it was probably helpful tobe like “oh yeah this thing I’m experiencing is that thing that Anna Salamon, the legendary wise rationalist calls PR.” Sort of ironic, I suppose. But I wouldn’t be surprised if young/new rationalists benefit a lot from seeing some high-status or high-wisdom rationalist write a post that describes a problem they experience. * Note that I think this also applies to many posts in Replacing Guilt & The Sequences. To have Eliezer Yudkowsky describe a problem you face not only helps you see it; it also helps you be like ah yes, that’s a real/important problem that smart/legitimate people face.  * The post “aged well.” It seems extremely relevant right now (Jan 2023), both for collectives and for individuals. The EA community is dealing with a lot of debate around PR right now. Also, more anecdotally, the Bay Area AI safety scene has quite a strange Status Hierarchy Thing going on, and I think this is a significant barrier to progress. (One might even say that “feeling afraid to speak openly due to vague social pressures” is a relatively central problem crippling the world at scale, as well as our community.) * The post is so short!   What could have been improved  * The PR frame. “PR” seems like a term that applies to organizations but not individuals. I think Anna could have pretty easily thrown in some more synonyms/near-synonyms that help people relate more
16Ben Pace
I'm not sure I use this particular price mechanism fairly often, but I think this post was involved in me moving toward often figuring out fair prices for things between friends and allies, which I think helps a lot. The post puts together lots of the relevant intuitions, which is what's so helpful about it. +4
19Elizabeth
I still think this is basically correct, and have raised my estimation of how important it is in x-risk in particular.  The emphasis on doing The Most Important Thing and Making Large Bets push people against leaving slack, which I think leads to high value but irregular opportunities for gains being ignored.
14orthonormal
The LW team is encouraging authors to review their own posts, so: In retrospect, I think this post set out to do a small thing, and did it well. This isn't a grand concept or a vast inferential distance, it's just a reframe that I think is valuable for many people to try for themselves. I still bring up this concept quite a lot when I'm trying to help people through their psychological troubles, and when I'm reflecting on my own. I don't know whether the post belongs in the Best of 2018, but I'm proud of it.
10Daniel Kokotajlo
This is one of those posts, like "when money is abundant, knowledge is the real wealth," that combines a memorable and informative and very useful and important slogan with a bunch of argumentation and examples to back up that slogan. I think this type of post is great for the LW review. I haven't found this advice super applicable to my own life (because I already generally didn't do things that were painful...) but it has found application in my thinking and conversation with friends. I think it gets at an important phenomenon/problem for many people and provides a useful antidote.
17alkjash
This feels like an extremely important point. A huge number of arguments devolve into exactly this dynamic because each side only feels one of (the Rock|the Hard Place) as a viscerally real threat, while agreeing that the other is intellectually possible.  Figuring out that many, if not most, life decisions are "damned if you do, damned if you don't" was an extremely important tool for me to let go of big, arbitrary psychological attachments which I initially developed out of fear of one nasty outcome.
28Valentine
I thought I'd add a few quick notes as the author. As I reread this, a few things jump out for me: * I enjoy its writing style. Its clarity is probably part of why it was nominated. * I'd now say this post is making a couple of distinct claims: * External forces can shape what we want to do. (I.e., there are lotuses.) * It's possible to notice this in real time. (I.e., you can notice the taste of lotuses.) * It's good to do so. Otherwise we find our wanting aligned with others' goals regardless of how they relate to our own. * If you notice this, you'll find yourself wanting to spit out lotuses that you can tell pull you away from your goals. * I still basically agree with the content. * I think the emotional undertone is a little confused, says the version of me about 19 months later. That last point is probably the most interesting to meta-reviewers, so I'll say a little about that here. The basic emotional backdrop I brought in writing this was something like, "Look out, you could get hijacked! Better watch out!" And then luckily there's this thing you can be aware of, to defend yourself against one more form of psychic/emotional attack. Right? I think this is kind of nuts. It's a popular form of nuts, but it's still nuts. Looking at the Duolingo example I gave, it doesn't address the question of why those achievements counted as a lotus structure for me. There are tons of things others find have lotus nature that I don't (e.g., gambling). And vice versa: my mother (who's an avid Duolingo user) couldn't care less about those achievements. So what gives? I have a guess, but I think that's outside the purview of the purpose of these reviews. I'll just note that "We're in a worldwide memetic war zone where everyone is out to get us by hijacking our minds!" is (a) not the hypothesis to default to and (b) if true is itself a questionable meme that seems engineered to stimulate fight-or-flight type reactions that do, indeed, hijack clarity
23Scott Alexander
I was surprised that this post ever seemed surprising, which either means it wasn't revolutionary, or was *very* revolutionary. Since it has 229 karma, seems like it was the latter. I feel like the same post today would have been written with more explicit references to reinforcement learning, reward, addiction, and dopamine. The overall thesis seems to be that you can get a felt sense for these things, which would be surprising - isn't it the same kind of reward-seeking all the way down, including on things that are genuinely valuable? Not sure how to model this.
10Raemon
Partial Self Review: There's an obvious set of followup work to be done here, which is to ask "Okay, this post was vague poetry meant to roughly illustrate a point. But, how many words do you actually precisely have?" What are the in-depth models that let you predict precisely how much nuance you have to work with? Less obvious to me is whether this post should become a longer, more rigorous post, or whether it should stay it's short, poetic self, and have those questions get explored in a different post with different goals.  Also less obvious to me is how the LessWrong Review should relate to short, poetic posts. I think it's quite important that this post be clearly labeled as poetry, and also, that we consider the work "unfinished" until there is a some kind of post that delves more deeply into these questions. But, for example, I think Babble last year was more like poetry than like a clear model, and it was nonetheless valuable and good to be part of the Best Of book. So, I'm thinking about this post from two lenses.  1. What are simple net-improvements I can make to this post, without sacrificing it's overall aim of being short/accessible/poetic? 2. Sketch out the research/theory agenda I'd want to see for the more detailed version. I did just look over the post, and notice that the poetry... wasn't especially good. There is mild cleverness in having the sections get shorter as they discuss larger coordination-groups. But I think I probably could write a post that was differently poetic. Or, find ways of making a short/accessible version that doesn't bother being poetic but is is nonetheless clear. I'm worried about every post having to be a rigorous, fully caveated explanation. That might be the right standard for the review, but not obviously. Some points that should be made somewhere, whether editing the OP or in a followup: 1. Yes, you can invest in processes that help you reach more people, more reliably. But those tools are effortful to build
19AprilSR
While the concept that looking at the truth even when it hurts is important isn't revolutionary in the community, I think this post gave me a much more concrete model of the benefits. Sure, I knew about the abstract arguments that facing the truth is valuable, but I don't know if I'd have identified it as an essential skill for starting a company, or as being a critical component of staying in a bad relationship. (I think my model of bad relationships was that people knew leaving was a good idea, but were unable to act on that information—but in retrospect inability to even consider it totally might be what's going on some of the time.)
15A Ray
I read this sequence and then went through the whole thing.  Without this sequence I'd probably still be procrastinating / putting it off.  I think everything else I could write in review is less important than how directly this impacted me. Still, a review: (of the whole sequence, not just this post) First off, it signposts well what it is and who it's for.  I really appreciate when posts do that, and this clearly gives the top level focus and whats in/out. This sequence is "How to do a thing" - a pretty big thing, with a lot of steps and branches, but a single thing with a clear goal. The post is addressing a real need in the community (and it was a personal need for me as well) -- which I think are the best kinds of "how to do a thing" posts. It was detailed and informative while still keeping the individual points brief and organized. It specifically calls out decision points and options, how much they matter, what the choices are, and information relevant to choosing.  This is a huge energy-saver in terms of actually getting people to do this process. When I went through it, it was accurate, and I ran into the decision points and choices as expected. Extra appreciation for the first post which also includes a concrete call to action for a smaller/achievable-right-now thing for people to do (sign a declaration of intent to be cryopreserved).  Which I did!  I also think that a "thing you can do right now" is a great feature to have in "how to do a thing" posts. I'm in the USA, so I don't have much evaluation or feedback on how valuable this is to non-USA folks.  I really do appreciate that a bunch of extra information was added for non-USA cases, and it's organized such that it's easy to read/skim past if not needed. I know that this caused me personally to sign up for cryonics, and I hope others as well.  Inasmuch as the authors goal was for more people in our community to sign up for cryonics -- I think that's a great goal and I think they succeeded.
43Max H
My wife completed two cycles of IVF this year, and we had the sequence data from the preimplantation genetic testing on the resulting embryos analyzed for polygenic factors by the unnamed startup mentioned in this post. I can personally confirm that the practical advice in this post is generally excellent. The basic IVF + testing process is pretty straightforward (if expensive), but navigating the medical bureaucracy can be a hassle once you want to do anything unusual (like using a non-default PGT provider), and many clinics aren't going to help you with anything to do with polygenic screening, even if they are open to it in principle. So knowing exactly what you want and what you need to ask for is key. Since this post was written, there have been lots of other developments and related posts in this general area: * Significantly Enhancing Adult Intelligence With Gene Editing May Be Possible * Superbabies: Putting The Pieces Together * Gameto Announces World’s First Live Birth Using Fertilo Procedure that Matures Eggs Outside the Body * Overview of strong human intelligence amplification methods: Genomic approaches And probably many others I am forgetting. But if you're a prospective parent looking for practical advice on how to navigate the IVF process and take advantage of the latest in genetic screening technology, this post is still the best place to start that I know of. Some of the things in the list above are more speculative, but the technology for selection is basically ready and practical now, and the effect size doesn't have to be very large for it to beat the status quo of having an embryologist eyeball it. I think this post is a slam dunk for a +9 and a spot in the LW canon, both for its object-level information and its exemplary embodiment of the virtue of empiricism and instrumental rationality. The rest of this review details my own experience with IVF in the U.S. in 2024. ---------------------------------------- This section of the orig
33DanielFilan
I think this post, as promised in the epistemic status, errs on the side of simplistic poetry. I see its core contribution as saying that the more people you want to communicate to, the less you can communicate to them, because the marginal people aren't willing to put in work to understand you, and because it's harder to talk to marginal people who are far away and can't ask clarifying questions or see your facial expressions or hear your tone of voice. The numbers attached (e.g. 'five' and 'thousands of people') seem to not be super precise. That being said: the numbers are the easiest thing to take away from this post. The title includes the words 'about five' but not the words 'simplifed poetry'. And I'm just not sure about the numbers. The best part of the post is the initial part, which does a calculation and links to a paper to support an order-of-magnitude calculation on how many words you can communicate to people. But as the paragraphs go on, the justifications get less airtight, until it's basically an assertion. I think I understand stylistically why this was done, but at the end of the day that's the trade-off that was made. So a reader of this post has to ask themselves: Why is the number about five? Is this 'about' meaning that you have a factor of 2 wiggle-room? 10? 100? How do I know that this kicks in once I hit thousands of people, rather than hundreds or millions? If I want to communicate to billions of people, does that go down much? These questions are left unanswered in the post. That would be fine if they were answered somewhere else that was linked, but they aren't. As such, the discerning reader should only believe the conclusion (to the extent they can make it out) if they trust Ray Arnold, the author. I think plausibly people should trust Ray on this, at least people who know him. But much of the readership of this post doesn't know him and has no reason to trust him on this one. Overall: this post has a true and important core that c
26Screwtape
Figuring out the edge cases about honesty and truth seem important to me, both as a matter of personal aesthetics and as a matter for LessWrong to pay attention to. One of the things people have used to describe what makes LessWrong special is that it's a community focused on truth-seeking, which makes "what is truth anyway and how do we talk about it" a worthwhile topic of conversation. This article talks about it, in a way that's clear. (The positive example negative example pattern is a good approach to a topic that can really suffer from illusion of transparency.) Like Eliezer's Meta-Honesty post, the approach suggested does rely on some fast verbal footwork, though the footwork need not be as fast as Meta-Honesty. Passing the Onion Test consistently requires the same kind of comparison to alternate worlds as glomarization, which is a bit of a strike against it but that's hardly unique to the Onion Test. I don't know if people still wind up feeling mislead? For instance, I can imagine someone saying "I usually keep my financial state private" and having their conversation partners walk away with wildly different ideas of how they're doing. Is it so bad they don't want to talk about it? Is it so good they don't want to brag? If I thought it was the former and offered to cover their share of dinner repeatedly, I might be annoyed if it turns out to be the latter. I don't particularly hold myself to the Onion Test, but it did provide another angle on the subject that I appreciated. Nobody has yet used it this way around me, but I could also see Onion Test declared in a similar manner to Crocker's Rules, an opt-in social norm that might be recognized by others if it got popular enough. I'm not sure it's worth the limited conceptual slots a community can have for those, but I wouldn't feel the slot was wasted if Onion Tests made it that far. This might be weird, but I really appreciate people having the conversations about what they think is honest and in what way
10Alex_Altair
Earlier this year I spent a lot of time trying to understand how to do research better. This post was one of the few resources that actually helped. It described several models that I resonated with, but which I had not read anywhere else. It essentially described a lot of the things I was already doing, and this gave me more confidence in deciding to continue doing full time AI alignment research. (It also helps that Karnofsky is an accomplished researcher, and so his advice has more weight!)
11adamShimi
This post proposes 4 ideas to help building gears-level models from papers that already passed the standard epistemic check (statistics, incentives): * Look for papers which are very specific and technical, to limit the incentives to overemphasize results and present them in a “saving the world” light. * Focus on data instead of on interpretations. * Read papers on different aspects of the same question/gear * Look for mediating variables/gears to explain multiple results at once (The second section, “Zombie Theories”, sounds more like epistemic check than gears-level modeling to me) I didn’t read this post before today, so it’s hard to judge the influence it will have on me. Still, I can already say that the first idea (move away from the goal) is one I had never encountered, and by itself it probably helps a lot in literature search and paper reading. The other three ideas are more obvious to me, but I’m glad that they’re stated somewhere in detail. The examples drawn from biology also definitely help.
11pjeby
I got an email from Jacob L. suggesting I review my own post, to add anything that might offer a more current perspective, so here goes... One thing I've learned since writing this is that counterfactualizing, while it doesn't always cause akrasia, it is definitely an important part of how we maintain akrasia: what some people have dubbed "meta-akrasia". When we counterfactualize that we "should have done" something, we create moral license for our past behavior. But also, when we encounter a problem and think, "I should [future action]", we are often licensing ourselves to not do something now. In both cases, the real purpose of the "should" in our thoughts is to avoid thinking about something unpleasant in the current moment. Whether we punish our past self or promote our future self, both moves will feel better than thinking about the actual problem... if the problem conflicts with our desired self-image. But neither one actually results in any positive change, because our subconscious intent is to virtue-signal away the cognitive dissonance arising from an ego threat... not to actually do anything about the problem from which the ego threat arose. In the year since I wrote this article, I've stopped viewing the odd things people have to be talked out of (in order to change) as weird, individual, one-off phenomena, and begun viewing them in terms of "flinch defenses"... which is to say, "how people keep themselves stuck by rationalizing away ego threats instead of addressing them directly." There are other rationalizations besides counterfactual ones, of course, but the concepts in this article (and the subsequent discussion in comments) helped to point me in the right direction to refine the flinch-defense pattern as a specific pattern and category, rather than as an ad hoc collection of similar-but-different behavior patterns.