A Map that Reflects the Territory

The best LessWrong essays from 2018, in a set of physical books

A beautifully designed collection of books, each small enough to fit in your pocket. The book set contains over forty chapters by more than twenty authors including Eliezer Yudkowsky and Scott Alexander. This is a collection of opinionated essays exploring argument, aesthetics, game theory, artificial intelligence, introspection, markets, and more, as part of LessWrong's mission to understand the laws that govern reasoning and decision-making, and build a map that reflects the territory.

Learn More

Recent Discussion

Science aims to come up with good theories about the world - but what makes a theory good? The standard view is that the key traits are predictive accuracy and simplicity. Deutsch focuses instead on the concepts of explanation and understanding: a good theory is an explanation which enhances our understanding of the world. This is already a substantive claim, because various schools of instrumentalism have been fairly influential in the philosophy of science. I do think that this perspective has a lot of potential, and later in this essay explore some ways to extend it. First, though, I discuss a few of Deutsch's arguments which I don't think succeed, in particular when compared to the bayesian rationalist position defended by Yudkowsky.

To start, Deutsch says that good...

The issues are whether the quantity, which you have called a probability , actually is probability , and whether the thing you at treating as a model of reality, is actually a such a model , in the sense of scientific realism, or merely something that churns out predictions, in the sense of instrumentalism.

I’m not quite sure how to respond to this; like, I think you’re right that SI is not solving the hard problem, but I think you’re wrong that SI is not solving the easy problem.

What are the hard and easy problems? Realism and instrumentalism? I ha... (read more)


This was a triumph
I'm making a note here, huge success

No, seriously, it was awful. I deleted my blog of 1,557 posts. I wanted to protect my privacy, but I ended up with articles about me in New Yorker, Reason, and The Daily Beast. I wanted to protect my anonymity, but I Streisand-Effected myself, and a bunch of trolls went around posting my real name everywhere they could find. I wanted to avoid losing my day job, but ended up quitting so they wouldn't be affected by the fallout. I lost a five-digit sum in advertising and Patreon fees. I accidentally sent about three hundred emails to each of five thousand people in the process of trying to put my blog back up.

I had, not to mince words about it, a really weird year.

The first post on Scott Alexander's new blog on Substack, Astral Codex Ten.

Good news : slate star codex is up again.
Bad news : I've been singing "still alive" since this morning and it's driving me crazy.

4Baisius1hDoes Scott's contract with Substack prevent automatic cross-posting here? I really do loathe Substack as a UI.
2Dagon40mInoreader lets me subscribe to the feed (URL https://astralcodexten.substack.com/feed/, [https://astralcodexten.substack.com/feed/,] which looks like standard RSS to me), so it doesn't seem that Substack is intentionally limiting access to their site.
1Dirichlet-to-Neumann8hThat's some great news !

Aside from worries over the new strains, I would be saying this was an exceptionally good week.

Both deaths and positive test percentages took a dramatic turn downwards, and likely will continue that trend for at least several weeks. Things are still quite short-term bad in many places, but things are starting to improve. Even hospitalizations are slightly down. 

It is noticeably safer out there than it was a few weeks ago, and a few weeks from now will be noticeably safer than it is today. 

Studies came out that confirmed that being previously infected conveys strong immunity for as long as we have been able to measure it. As usual, the findings were misrepresented, but the news is good. I put my analysis here in a distinct post, so...

1bardstale6hWearing a mask after vaccination would reduce the spread of other diseases such as the flu, thus freeing additional healthcare resources for COVID-19 patients.
1TheMajor9hIt was pointed out to me that it is really not accurate to consider the UK daily COVID numbers as a single data-point. There could be any number of possible explanations for the decrease in the numbers. Some possible explanations include: 1. The current lockdown and measures are sufficient to bring the English variant to R<1. 2. The current measures bring the English variant to an R slightly above 1, and the wild variants to R well below 1, and because nationally the English variant is not dominant yet (even though it is in certain regions) this gives a national R<1. 3. The English strain has spread so aggressively regionally that group immunity effects in the London area have significantly slowed the spread, while not spreading as quickly geographically. Most notably, hypotheses 2 & 3 predict that the stagnation will soon reverse back into acceleration (with hypothesis 3 predicting a far higher rate than 2), as the English variant becomes more prevalent throughout the rest of the UK. Let's hope the answer is door number 1?
1MondSemmel10hHave you seen VaccinateCA [https://www.vaccinateca.com/], a volunteer effort that hopes to help CA citizens make sense of the mess in that state?

I keep thinking about how if at any point we were all able to actually quarantine for two weeks1 at the same time, the pandemic would be over.

Like, if instead of everyone being more or less cautious over a year, we all agreed on single a two week period to hard quarantine. With plenty of warning, so that people had time to stock up on groceries and do anything important ahead of time. And with massive financial redistribution in advance, so that everyone could afford two weeks without work. And with some planning to equip the few essential-every-week-without-delay workers (e.g. nurses, people keeping the power on) with unsustainably excessive PPE.

This wouldn’t require less total risky activity. If we just managed to move all of the risky activity...

Since the virus grows exponentially (when R>1), and it seems unlikely that it'll be actually eradicated (due to outliers on infection times, groups who isolate together, but slowly transmit it among the group, and simple leaks in enforcement/adoption), it's best to think of this kind of intervention as "how much does it reduce the infectious population"?  

It seems likely that, with the vaccines rolling out (too slowly, but happening), any significant reduction in the spread makes the final herd immunity (actual end-game for this) contain a higher r... (read more)

4lsusr32mThis could work in theory but to separate everyone you have to (separate everyone, including the elderly from their caretakers, mothers from their babies, comatose patients from doctors and prisoners from each other—all at the same time. Plus animals, as Bucky mentioned. Fortunately, quarantining everyone is overkill. It is more efficient to do contact tracing and testing, and then quarantine only those who test positive or were exposed to someone who tested positive. Countries like Taiwan did this successfully. They quarantine everyone entering the country and have had the pandemic under control for ages. You can walk around Taipei like it's 2018.
10Elizabeth1hFrom research [https://roamresearch.com/#/app/AcesoUnderGlass/page/RSnBZ6tdd] I did on New Zealand: 14 days isn't long enough if people are isolating together- live infections get passed to housemates partway through, who are still contagious when the 14 days are up. So either you have to truly isolate everyone, or extend it by some number of days for each additional person. Also, I assume you mean "14 days, unless they test positive or show symptoms", but then you have to figure out how to test and verify symptoms for everyone.
2ryan_b2hI agree that you have identified the key problems. 1 and 2 appear to me so unsolvable as to be indistinguishable from impossible, and any other issue I can think of is gated through one of them. I have an extremely strong prior that any plan which requires an entire population to change their behavior at once is fundamentally wrong and not worth considering. Although I do note that plans involving a fraction of the population changing their behavior, or an entire population changing their behavior over time, are still worth considering.

Cross-posted from Living Within Reason

In philosophy, the Principle of Charity is a technique in which you evaluate your opponent’s position as if it made the most amount of sense possible given the wording of the argument. That is, if you could interpret your opponent’s argument in multiple ways, you would go for the most reasonable version.

– UnclGhost

There is a corollary to the Principle of Charity which I’m calling Bayesian Charity. It says that, in general, you should interpret your opponent’s wording to be advocating the most popular position.

This is implied by Bayes Theorem. Your prior for whether someone believes something unpopular should be related to its popularity. Since an unpopular belief, by definition, has few believers, your prior for whether someone believes an unpopular position should be lower the...

Wait.  Interpreting by (your perception of) popularity seems to be less truth-seeking than the most sensible OR that strongest (see https://www.lesswrong.com/tag/steelmanning ).  

If you're talking about using it as evidence that at least one person makes this claim, then I see your point - you should consider all possible interpretations of an argument, weighted by likelihood that the person is making that specific claim.  That likelihood is subject to Bayesean calculations.  "popularity" is a fine prior, but you ALWAYS have additional evidence to alter your weighting of probability from that point.

Which variables are most important for predicting and influencing how AI goes?

Here are some examples:

  • Timelines: “When will crazy AI stuff start to happen?”
  • Alignment tax: “How much more difficult will it be to create an aligned AI vs an unaligned AI when it becomes possible to create powerful AI?
  • Homogeneity: "Will transformative AI systems be trained/created all in the same way?"
  • Unipolar / Multipolar: "Will transformative AI systems be controlled by one organization or many?"
  • Takeoff speeds: "Will takeoff be fast or slow (or hard or soft, etc.)?"

We made this question to crowd-source more entries for our list, along with operationalizations and judgments of relative importance. This is the first step of a larger project.


  1. Answers should be variables that are importantly different from the previous answers. It’s OK
Answer by steve2152Jan 22, 20212Ω1

public sympathy vs dehumanization? ... Like, people could perceive AI algorithms as they do now (just algorithms), or they could perceive (some) AI algorithms as deserving of rights and sympathies like they and their human friends are. Or other possibilities, I suppose. I think it would depend strongly on the nature of the algorithm, as well as on superficial things like whether there are widely-available AI algorithms with cute faces and charismatic, human-like personalities, and whether the public even knows that the algorithm exists, as well as random t... (read more)

3Answer by Noa Nabeshima43mValue-symmetry: "Will AI systems in the critical period be equally useful for different values?" This could fail if, for example [https://www.alignmentforum.org/posts/HBxe6wdjxK239zajf/what-failure-looks-like] , we can build AI systems that are very good at optimizing for easy-to-measure values but significantly worse at optimizing for hard to measure values. It might be easy to build a sovereign AI to maximize the profit of a company, but hard to create one that cares about humans and what they want. Evan Hubinger has some operationalizations of things like this here [https://www.lesswrong.com/posts/jGB7Pd5q8ivBor8Ee/impact-measurement-and-value-neutrality-verification-1] andhere [https://www.lesswrong.com/posts/WwJdaymwKq6qyJqBX/operationalizing-compatibility-with-strategy-stealing] .
2Answer by Daniel Kokotajlo1hCraziness: "Will the world be weird and crazy in the crucial period?" For example, are lots of important things happening fast, such that it's hard to keep up without AI assistance, is the strategic landscape importantly different from what we expected thanks to new technologies and/or other developments [https://aiimpacts.org/relevant-pre-agi-possibilities/], does the landscape of effective strategies for AI risk reducers [https://www.lesswrong.com/posts/zjhZpZi76kEBRnjiw/relevant-pre-agi-possibilities#Human_effectiveness] look importantly different [https://www.lesswrong.com/posts/D2AgH7wNEz2SJCgBs/better-name-for-heavy-tailedness-of-the-world] than it does now...
3Answer by Daniel Kokotajlo1hRisk Awareness: “In the critical period, will it be widely believed by most of the relevant people that AI is a serious existential risk?” This is closely related to whether or not there are warning shots [https://www.lesswrong.com/posts/hLKKH9CM6NDiJBabC/what-are-the-most-plausible-ai-safety-warning-shot-scenarios] or fire alarms [https://intelligence.org/2017/10/13/fire-alarm/], but in principle it could happen without any of either.

Boxed Topics, Jenga Towers, And The Spacing Effect.

An undergraduate class on molecular biology teaches you about DNA transcription, the Golgi apparatus, cancer, and integral membrane proteins. Sometimes, these sub-topics are connected. But most often, they're presented in separate chapters, each in its own little box. So let's call these Boxed Topics.

The well-known Stewart calculus textbook teaches you about functions in chapter 1, limits and the definition of derivatives in chapter 2, rules for derivatives in chapter 3, and the relationship of derivatives with graphs in chapter 4. Woe betide you if you weren't entirely clear on the definition of a derivative when it gets used, over and over again, in next week's proofs of derivative rules.

Taking a calculus class can be like building a Jenga Tower. If...

"Beginners in college-level math would learn about functions, the basics of linear systems, and the difference between quantitative and qualitative data, all at the same time."

This seems to be the standard approach for undergraduate-level mathematics at university, at least in Europe. 


If data keeps coming out in the next week confirming that the new COVID strain is 70% more transmissible, I think the modal outcome is that ~50% of Americans will get it by the early summer. The market may take a few days to realize and react to this (as it was in March), but also just buying June put option on the SP500 seems very naïve (since the SP500 is at all-time highs and a fourth COVID wave doesn't necessarily affect much the NPV of future earning of huge corporations). So if I think that the probability of everyone getting COVID in the next six months is much likelier than the market, at least for a few days, what trade would capture that?

Presumably short-term, e.g. expiring Feb/March. (FYI I've just bought lots of VIX Feb & Mar futures.)

Psycho-Pass takes place in a cyberpunk dystopia ruled by a totalitarian AI dictator. Cyberpunk stories are often about evading the law. What makes Psycho-Pass special is its protagonist is a police officer.

Tsunemori Akane's job is to suppress crime. This involves suppressing violent criminals, which is a good thing. The AI's surveillance state makes it possible to suppress crime before it happens, which is even better. Potential criminals often include punks, radicals, gays, artists, musicians, visionaries and detectives which is…

Wait a minute.


If Psycho-Pass was written in America then Tsunemori's character arc would be a journey of disillusion. She would be commanded to do something unethical. Tsunemori would refuse. Her valiant act of disobedience would instigate a cascade of disorder leading to a revolution and the eventual overthrew...

6Vanilla_cabs4hShame really, we are still short of one for our Evil Psychopath monthly poker night! Let us know if you manage to acquire the evil of psychopathy you're missing before the end of the month! We've got smoothies! That's one difference I regularly meet between western stories and anime. In the West, evil antagonists seem reduced to two qualities: they're bad, and we don't want/need to know about them. Evil here is like mysterious [https://www.lesswrong.com/posts/6i3zToomS86oj9bS6/mysterious-answers-to-mysterious-questions] . Conversely, in anime, most villains have their motives explained as well as heroes. Sometimes, it's the same motive! Typically, loyalty to friends (the °1 motive for heroes and villains alike in shonen). This makes villains much more interesting and relatable. Villains are not alien, of a different substance than us. They're like us, except they dare do what we don't, and in doing that they exemplify their values in a way that lets us explore counterfactuals and learn from that experience. They're so interesting that they can become more popular than heroes (Yagami Light). A show that does it masterfully is Attack on Titan. It's excellent at circling a character's point of view in a few quick strokes and never making light of it. I can sympathise with really any character, however beef they have among themselves.

Psychopathically implementing our own values is a good way to put it. Now that you frame it this way, I can't think of a good anime whose villains don't have well-fleshed out motives. This is in stark contrast to Marvel, Star Wars, etc. where the villains' ideologies fell like caricatures.

1just_browsing1h(this is just a rant, not insightful) Everybody knows how important it is to choose the right time to write something. The optimal time is when you're really invested in the topic, learning rapidly but know enough to start the writing process. Then, ideally, during the writing process everything will crystalize. If you wait much longer than this the topic will no longer be exciting and you will not want to write about it. Everybody gives this advice, both within and outside of academia. I've heard it from professors, LW-y blog posts (maybe even on LW?), and everywhere in between. SO WHY DO I CONSTANTLY IGNORE THIS ADVICE?? :(

This isn't a direct answer to your question, but what I've personally found is that if I want to get re-excited about a topic that has already passed that critical period, the best thing to do is find people either asking questions about it or Being Wrong On The Internet about it, so that then I want to explain or rant about it again. ;-)