X

A Map that Reflects the Territory

The best LessWrong essays from 2018, in a set of physical books

A beautifully designed collection of books, each small enough to fit in your pocket. The book set contains over forty chapters by more than twenty authors including Eliezer Yudkowsky and Scott Alexander. This is a collection of opinionated essays exploring argument, aesthetics, game theory, artificial intelligence, introspection, markets, and more, as part of LessWrong's mission to understand the laws that govern reasoning and decision-making, and build a map that reflects the territory.

Hide
Learn More

Recent Discussion

So, if you've been reading my Less Wrong posts over the last twenty-one months, you'll notice that I've been doing a lot of philosophy-of-language blogging lately! Those posts are:

But, if you didn't know, the reason I've been doing so much philosophy-of-language blogging lately is because I was frustrated because I thought a lot of people were motivatedly getting the philosophy of language wrong for political convenience, specifically around transgender issues. (I think assertions like "trans women are women" and similar need to be argued for on the empirical merits; you can't just define them to be true.)

In particular, "Where to Draw the Boundaries?"...

If I understand it correctly, you are willing to use the preferred pronouns for passing trans people, because your System 1 doesn't object. May I interpret it as: "as long as we are not mentioning biology explicitly, I am okay to use the words 'man/woman' in the sense 'what my System 1 perceives as a man/woman', because that is a natural boundary"? (Like, biology is one natural boundary, System 1 judgment in another natural boundary, neither is inherently better than the other, the proper choice depends on context.)

This makes me imagine an opposite example... (read more)

Much has been written about the fundamental opposition between explore and exploit, chaos and order, yin and yang. In this post I make two observations about the psychology of this opposition.

In the first part, I challenge the metaphor of the comfort zone: a slowly-changing region in activity-space where everything inside is comfortable and everything outside induces anxiety. The point is that anxiety depends not only on the spookiness of the activity itself, but on one's proximity to safety. I am afraid because it is dark; I am terrified because the light switch is all the way down the hall. In particular, it is possible to reduce anxiety by bringing your comfort zone with you in the form of a safety behavior or a trusted companion. This serves as an alternative to Comfort Zone Expansion.

In the second part, I note that explore and exploit are often embodied in the human personality...

I love your writing. That is all.

"Gnomish helms should not function.  Their very construction seems to defy the nature of thaumaturgical law.  In fact, they are impossible.  Like most products of gnomish minds, they include a large number of bells and whistles, and very little substance.  Those that work usually have a minor helm contained within, always hidden away, disguised to appear innocuous and inessential."
              -- Spelljammer campaign set

We have seen that knowledge implies mutual information between a mind and its environment, and we have seen that this mutual information is negentropy in a very physical sense:  If you know where molecules are and how fast they're moving, you can turn heat into work via a Maxwell's Demon / Szilard engine.

We have seen that forming true beliefs without evidence is the same sort of improbability as a hot glass of water spontaneously reorganizing into ice cubes and electricity.  Rationality takes "work" in a thermodynamic sense, not just the sense of mental effort;...

C is Turing-complete, which means Gödel-complete, so yeah, the universe can be viewed as a C program.

I was chatting with friends about the Covid vaccine and they were concerned about side effects. I don’t expect significant side effects but don’t really have a principled analysis as to why not.

What kind of probability should one assign to significant side effects from a vaccine, particularly long term side effects which are less likely to be picked up in shorter testing regimes?

Comparing to other vaccines is helpful. But what about a more outside view of new medical treatments? I'm not sure what the reference class should be, but I think the fact that the mRNA vaccine has never been used before should give us pause.

In this essay, I will argue that the analysis of Sandberg et al. is flawed in a number of key respects, and as a result the Fermi Paradox remains an open question. Here I briefly list the key problems with the Sandberg et al. paper, before proceeding to discuss each in more detail.

  1. The method used of multiplying uncertainties of many small numbers, most of which have an upper bound of one, is biased towards yielding a result of a high probability of Earth being unique, while also leading to various dubious results.
  2. The key result of the paper is driven largely by uncertainty in the parameter fl, which is modeled in an unusual way without clear justification.
  3. Adoption of slightly different (and I believe more plausible) modelling choices and parameter values yields totally different results, which do not result in the Fermi paradox being dissolved. I illustrate this by re-estimating the Sandberg et al. models using different parameters and modelling assumptions.

abiogenesis being so early on Earth is 100% survivorship bias

Being early on earth was not necessary for survival. Similarly, being early for the formation of stars of suitable temperatures also wasn't especially favored by anthropics. Neither of those things had to happen.

I didn’t learn about history very well prior to my thirties somehow, but lately I’ve been variously trying to rectify this. Lately I’ve been reading Howard Zinn’s People’s History of the United States, listening to Steven Pinker’s the Better Angels of Our Nature, watching Ken Burns and Lynn Novick’s documentary about the Vietnam War and watching Oversimplified history videos on YouTube (which I find too lighthearted for the subject matter, but if you want to squeeze extra history learning in your leisure and dessert time, compromises can be worth it.)

There is a basic feature of all this that I’m perpetually confused about: how has there been so much energy for going to war?

It’s hard to explain my confusion, because in each particular case, there might be plenty of plausible motives given–someone wants ‘power’, or to ‘reunite their country’, or there is some customary enemy, or that enemy might attack them...

Is the question why tribes go to war, or or why individuals do?

The first question isn't hard to answer. For most of history, there weren't other means of conflict resolution , there weren't other means of economic expansion , and it was a matter of life and death.

2ryan_b4hThis varies a lot based on the context, as you might expect. It is worth pointing out that a lot of the details which speak to your confusions are filtered out just because of the format in which you are consuming history; popular histories necessarily dispense with nuanced details behind popular opinion or decisions of rulers/governments/etc; they spare none at all for the individual soldiers. It is also worth pointing out that this is not even a little bit settled of a question; it doesn't even have rigorous schools of thought - we (the civilizational we) are only in the process of clarifying this now. That being said, I claim it is because of local incentives. The simplest one is brutally straightforward: most of history's soldiers were not volunteers but conscripts. In pre-modern times the bargain was this: come with the army or die immediately. Going to war, for the group of men who did, was mostly a matter of sparing their own lives and the lives of their village from their own army. In modern times we have refined this process considerably, but fundamentally it remains a matter of a highly probable harm up front versus a distant and uncertain harm in the future. Volunteers are more nuanced, but they still act according to local incentives. Most of time it is because the army offers more reliable food, shelter, and payment than they would get by staying where they are; maybe dying of dysentery on the march isn't as bad as definitely freezing or starving next winter. Since the development of the professional military the army is considered a career like any other, and gets put down next to doctor or engineer for officers, and welder or electrician for enlisted. Volunteers who join for moral reasons (like me) can probably still mostly be chunked as responding to local incentives, where local incentives includes social incentives. A lot of people in this group do it because it is a family tradition, or because it is highly respected in the area where they a
4ryan_b7hI endorse this recommendation, and am reading the book myself periodically.
6Daniel Kokotajlo7hI agree with Kaj; humans have evolved for war. Also, human culture has evolved for war. As a result, war is glorious and appealing, at least to many humans. (Think about how a huge fraction of video games involve fighting. Fighting is fun! Fighting is cool!) I remember reading a quote from a young soldier in Iraq: "War is better than sex!" On the less juvenile end of the spectrum, war is an adventure, and adventure is appealing. War is also a competition, and competition is appealing. War also is a great opportunity to demonstrate virtue, which means that lots of heroes and heroic stories happen in war, which means that when people look for role-models or fantasize about achieving status and glory, war features prominently.

Just wanted to add that "made me an easier agent to coordinate with" applies not only to coordination with other people, but also to coordination with your past/future selves. That is, what is "good for your soul" is good even when other people are not involved.

It may even be the more important aspect, because if you can't trust your future selves, how could other people? (Your deals with other people implicitly involve deals with your future selves.)

4elityre12hI think humans have souls. It just so happens that they aren't immortal by default. I wouldn't want to make your substitution, for the same reason why Taleb doesn't like the substitution of artificial formula for a mother's milk: the substitution implies an assumption that you've correctly understood everything important about the thing you're replacing. I bet there is more to a soul than what your long sentence gets at, and I don't want to cut out that "more" prematurely.
1jp9hI think you're gonna need to define soul here. Not in a way that implies you've understood everything, but in the way that you might describe fire as the red hot stuff.
2mr-hire8hThe soul is the metaphorical red hot stuff :D.

I started this post thinking that I would write all the rules for evaluating Jacobians of neural network parameters in specific cases. But while this would certainly be useful for grokking deep learning papers, frankly it's difficult to write that in Latex and the people who have written The Matrix Calculus You Need For Deep Learning paper have already done it much better than I can do.

Rather, I consider my comparative advantage here to provide some expansion on why we should use Jacobians in the first place. If you were to just read the paper above, you might start to think that Jacobians are just notational perks. I hope to convince you that they are much more than that. In at least one setting, Jacobians provide a mathematical framework for analyzing the input-output behavior of deep neural networks, which can help us see things which we might have missed without... ,

3blue1brown touches on that way of thinking about functions/derivatives in this video 

(very similar to Khan academy video linked in the article)
 

Real world examples of the Parable of Predict-O-Matic show that trust in predictive accuracy has the power to shape world events. Accuracy brings trust, and trust brings power.

It's therefore a bit surprising that more people don't publish verifiable forecasts about major world events, along the lines of Philip Tetlock's Superforecasting. Even if they were guessing randomly, a lucky streak might bring fame and fortune. Publishing your forecasts would be like buying a lottery ticket.

The explanation is that society defends itself against this threat by demanding that forecasters earn the right to predict. Hence, society's obsession with credentials, the struggle over who gets to grant them, and constant attempts to grab the right to predict by unlawful or deceptive means. By requiring that wannabe prognosticators earn a graduate degree before they're taken seriously, and by limit their forecasting privileges to their field of expertise, we hope to cut down on the...

The Blue Party says that a major Green Party policy will fail. When the Green Party enacts it, the Blues sabotage it, then use its failure as evidence that Blue ideology is correct.

Can you give an example? What is the difference between "sabotaging" a policy and simply opposing a policy you don't support?

I basically agree, but I do assign it to Moloch. *shrug