Find all Alignment Newsletter resources here. In particular, you can sign up, or look through this spreadsheet of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version here (may not be up yet).

Highlights

Self-training with Noisy Student improves ImageNet classification (Qizhe Xie et al) (summarized by Dan H): Instead of summarizing this paper, I'll provide an opinion describing the implications of this and other recent papers.

Dan H's opinion: Some in the safety community have speculate... (Read more)

In coherence arguments do not imply goal directed behavior Rohin Shah argues that a system's merely being at all model-able as an EU maximizer does not imply that it has "goal directed behavior". The argument as I understand it runs something like this:

1: Any behavior whatsoever maximizes some utility function.

2: Not all behaviors are goal directed.

Conclusion: A system's behavior maximizing some utility function does not imply that its behavior is goal directed.

I think this argument is technically sound, but misses an important connection between VNM coherence and goal dir... (Read more)

Stories such as Peter Singer's "drowning child" hypothetical frequently imply that there is a major funding gap for health interventions in poor countries, such that there is a moral imperative for people in rich-countries to give a large portion of their income to charity. There are simply not enough excess deaths for these claims to be plausible.

Much of this is a restatement of part of my series on GiveWell and the problem of partial funding, so if you read that carefully and in detail, this may not be new to you, but it's important enough to have its own concise post. Th... (Read more)

Let's say you don't like something about your local dance. Perhaps you'd like to see gender free calling, a different approach to booking bands, a new kind of special event, or something else. How can you make this happen?

The best case is that you talk to the organizers, they say "what a great idea!" and handle the rest, but it rarely works this way. [1] Maybe the organizers have a different background and don't understand why you think your ideas would be an improvement. Maybe your ideas would be more work, at least at first, and they're feeling overworked with what they're doing a... (Read more)

This article was originally a post on my tumblr. I'm in the process of moving most of these kinds of thoughts and discussions here.

Okay. There’s a social interaction concept that I’ve tried to convey multiple times in multiple conversations, so I’m going to just go ahead and make a graph.

I’m calling this concept “Affordance Widths”.

Let’s say there’s some behavior {B} that people can do more of, or less of. And everyone agrees that if you don’t do enough of the behavior, bad thing {X} happens; but if you do too much of the behavior, bad thing {Y} happens.

Now, let’s say we have five differ... (Read more)

One of the most pleasing things about probability and expected utility theory is that there are many coherence arguments that suggest that these are the “correct” ways to reason. If you deviate from what the theory prescribes, then you must be executing a dominated strategy. There must be some other strategy that never does any worse than your strategy, but does strictly better than your strategy with certainty in at least one situation. There’s a good explanation of these arguments here.

We shouldn’t expect mere humans to be able to notice any failures of coherence in a superintelligent agent,... (Read more)

I have gone to several events operating under the Chatham House Rule, and have overall found it more annoying than useful. In this post, I share why I dislike the rule, how I think it can be improved, and hopefully spark others to give ideas on how to improve it. In particular, I think the part about not revealing who was at the event should be opt in. Partially, my goal is to eventually develop a modified version that might be used at future events.

The Chatham House Rule states that "When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to use the i... (Read more)

Antimemes are self-keeping secrets. You can only perceive an antimeme if you already know it's there. Antimemes don't need a conspiracy to stay hidden because you can't comprehend an antimeme just by being told it exists. You can shout them to the heavens and nobody will listen. I'll try to explain with a fictitious example.

Suppose we all had an invisible organ behind our ears and our brains kept it secret from our consciousness. If I told you "you have an invisible organ behind your ear" you wouldn't believe me. You'd only believe it exists if you deduced its existence from a trail of evidenc

(cross posted on my roam blog)

There's an old Paul Graham Essay, "Keep Your Identity Small". It's short so it's worth it to read the whole thing right now if you've never seen it. The yisbifiefyb ("yeah it's short but i'm functionally illiterate except for your blog") is roughly "When something becomes part of your identity, you become dumber. Don't make things part of your identity."

I read that post some time in high school and thought, "Of course! You're so right Paul Graham. Cool, now I'll never identify as anyth... (Read more)

There’s an essay that periodically feels deeply relevant to a situation:

Someday I want to write a self-help book titled “F*k The Karate Kid: Why Life is So Much Harder Than We Think”.

Look at any movie with a training montage: The main character is very bad at something, then there is a sequence in the middle of the film set to upbeat music that shows him practicing. When it's done, he's an expert.

It seems so obvious that it actually feels insulting to point it out. But it's not obvious. Every adult I know--or at least the ones who are depressed--continually suffers from something like sticker

Without using eminent domain, a large chunk of the possible future value goes to surrounding land-owners who may have done little or nothing to create that value. It does not seem economically possible to build a city that is cheap to live in without locking the price of land down in some way, at some point. It is not obvious how to do this well, but eminent domain seems to be a necessary component of it. If even fairly rural land starts out pre-speculated, there is no hope, there is a value/livability value that cities cannot ever rise above.

Apparently, the Charter Cities Institute for all th... (Read more)

Elevator pitch: Bring enough light to simulate daylight into your home and office.

This idea has been shared in Less Wrong circles for a couple years. Yudkowsky wrote Inadequate Equilibria in 2017 where he and his wife invented the idea, and Raemon wrote a playbook in 2018 for how to do it yourself. Now I and at least two other friends are trying to build something similar, and I suspect there's a bigger-than-it-looks market opportunity here because it's one of those things that a lot of people would probably want, if they knew it existed and could experience it. And it's only recently become c

Yesterday:

There is this very, very old puzzle/observation in economics about the lawyer who spends an hour volunteering at the soup kitchen, instead of working an extra hour and donating the money to hire someone...

If the lawyer needs to work an hour at the soup kitchen to keep himself motivated and remind himself why he's doing what he's doing, that's fine.  But he should also be donating some of the hours he worked at the office, because that is the power of professional specialization and it is how grownups really get things done.  One might consider the check as buying the righ

Spanish has gramatical gender in a way English doesn't:

una amiga ruidosa — a loud (female) friend
un amigo ruidoso — a loud (male) friend
unas amigas ruidosas — some loud (female) friends
unos amigos ruidosos — some loud (not-all-female) friends

I remember when I was studying Spanish, learning the rule that even if you had a hundred girls and one boy you would use the male plural. My class all thought this was very sexist and unfair, but our teacher told us we were misapplying American norms and intuitions.

It's been interesting, ~twenty years later, following the developm... (Read more)

Consider the following program:

f(n):
    if n == 0:
        return 1
    return n * f(n-1)

Let’s think about the process by which this function is evaluated. We want to sketch out a causal DAG showing all of the intermediate calculations and the connections between them (feel free to pause reading and try this yourself).

Here’s what the causal DAG looks like:

Each dotted box corresponds to one call to the function f. The recursive call in f becomes a symmetry in the causal diagram: the DAG consists of an infinite sequence of copies of the same subcircuit.

More generally, we can represent any Tu

I was inspired by the APOD pictures and discussion here and here. The conditions for the 'experiment' are:

• For some mysterious reason, all the water on Earth (minus that in living things - the biosphere) suddenly & immediately converges into 17 identical sphere and let to drop down to the land below right away.
• The places of those water balls are the centers of 17 biggest tectonic plates as listed here.
• The bottoms of the huge balls just touch the highest points on the ground there. Also from my calculation, which may be wrong, their radius is 258 km.
• The ISS is currently at its hig

Let's start with a few examples (borrowed from here) to illustrate what we're talking about:

• We have a gas consisting of some huge number of particles. We throw away information about the particles themselves, instead keeping just a few summary statistics: average energy, number of particles, etc. We can then make highly precise predictions about things like e.g. pressure just based on the reduced information we've kept, without having to think about each individual particle. That reduced information is the "abstract layer" - the gas and its properties.
• We have a bunch o

[EDIT: A crucial consideration was pointed out in the comments. For all the designs I've looked at, it's cheaper to just get a heat exchanger and ventilation fans, and blow the air outside/pull it inside and eat the extra heating costs/throw on an extra layer of clothing, than it is to buy a CO2 stripper. There's still an application niche for poorly ventilated rooms without windows, but that describes a lot fewer occasions than my previous dreams of commercial use.]

So, I have finally completed building a CO2 stripper that removes CO2 from the air to (hopefully) improve cognitio... (Read more)

The most damaging thing you learned in school wasn't something you learned in any specific class. It was learning to get good grades.