This is very likely my most important idea, and I've been trying to write about it for years, but it's too important to write about it badly, so I haven't ever mustered the courage. I guess I'll just do a quick test round.

It starts with this "hierarchy of needs" model, first popularized my Maslow, that we humans tend to focus on one need at the time.

• If you're hungry, you don't care about aesthetics that much
• If you are competing for status, you can easily be tempted to defect on coordination problems
• etc

I like to model this roughly as an ensemble of subag... (Read more)

In How I do research, TurnTrout writes:

[I] Stare at the problem on my own, ignoring any existing thinking as much as possible. Just think about what the problem is, what's confusing about it, what a solution would look like. In retrospect, this has helped me avoid anchoring myself. Also, my prior for existing work is that it's confused and unhelpful, and I can do better by just thinking hard.

The MIRI alignment research field guide has a similar sentiment:

It’s easy to fall into a trap of (either implicitly or explicitly) conceptualizing “research” as “first studying and learning what’s al

I’m confused why people are so bad at dating. It seems to me like there are tons of $20 bills lying on the ground which no one picks up.

For example, we know that people systematically choose unattractive images for their dating profiles. Sites like PhotoFeeler cheaply (in some cases, freely) resolve this problem. Since photo quality is one of the strongest predictors of number of matches, you would think people would be clamoring to use these sites. And yet, not many people use them.

In the off-line dating world, it surprises me how few self-help books are about dating. Right now, zer... (Read more)

https://dominiccummings.com/2019/06/26/on-the-referendum-33-high-performance-government-cognitive-technologies-michael-nielsen-bret-victor-seeing-rooms/ (a)

By the architect of Brexit, who is now one of Boris Johnson's main advisors ("Special Adviser to the Prime Minister").

Cummings has a good amount of intellectual overlap with rationalist writers. (Check out his blogroll, in the sidebar on the right!)

From post's introduction:

This blog looks at an intersection of decision-making, technology, high performance teams and government. It sketches some ideas of physicist Mic

This was one of the most thought-provoking posts I read this month. Mostly because I spent a really large number of hours of my life sleeping, and also significantly increased the amount that I've been sleeping over the past three years, and this has me seriously considering reducing that number again. 

The opening section of the article: 

Matthew Walker (a) is a professor of neuroscience and psychology at the University of California, Berkeley, where he also leads the Center for Human Sleep Science.

His book Why We Sleep (a) was published in September 2017. Part survey of sleep r

Imagine a black box which, when you pressed a button, would generate a scientific hypothesis. 50% of its hypotheses are false; 50% are true hypotheses as game-changing and elegant as relativity. Even despite the error rate, it’s easy to see this box would quickly surpass space capsules, da Vinci paintings, and printer ink cartridges to become the most valuable object in the world. Scientific progress on demand, and all you have to do is test some stuff to see if it’s true? I don’t want to devalue experimentalists. They do great work. But it’s appropriate that Einstein i... (Read more)

One of the most pleasing things about probability and expected utility theory is that there are many coherence arguments that suggest that these are the “correct” ways to reason. If you deviate from what the theory prescribes, then you must be executing a dominated strategy. There must be some other strategy that never does any worse than your strategy, but does strictly better than your strategy with certainty in at least one situation. There’s a good explanation of these arguments here.

We shouldn’t expect mere humans to be able to notice any failures of coherence in a superintelligent agent,... (Read more)

Recently I started picking up French again. I remembered getting something out of Duolingo a few years ago, so I logged in.

Since the last time I was there, they added an “achievements” mechanic:

I noticed this by earning one. I think it was “Sharpshooter”. They gave me the first of three stars for something like doing five lessons without mistakes. In the “achievements” section, it showed me that I could earn the second star by doing twenty lessons in a row flawlessly.

And my brain cared.

I watched myself hungering to get the achievements. These arbitrary things that someone had just stuck on the... (Read more)

I’m working on a theory of abstraction suitable as a foundation for embedded agency and specifically multi-level world models. I want to use real-world examples to build a fast feedback loop for theory development, so a natural first step is to build a starting list of examples which capture various relevant aspects of the problem.

These are mainly focused on causal abstraction, in which both the concrete and abstract model are causal DAGs with some natural correspondence between counterfactuals on the two. (There are some exceptions, though.) The list isn’t very long; I’ve... (Read more)

Author's Note: This post is a bunch of mathy research stuff with very little explanation of context. Other posts in this sequence will provide more context, but you might want to skip this one unless you're looking for mathy details.

Suppose we have a medical sensor measuring some physiological parameter. The parameter has a constant true value $X$, and the sensor takes measurements $M_1\dots M_n$ over a short period of time. Each measurement has IID error (so the measurements are conditionally independent given $X$). In the end, the measurements are averaged together, and there’s a ... (Read more)

I intend to use my shortform feed for two purposes:

1. To post thoughts that I think are worth sharing that I can then reference in the future in order to explain some belief or opinion I have.

2. To post half-finished thoughts about the math or computer science thing I'm learning at the moment. These might be slightly boring and for that I apologize.

Sorry for the flashy headline, but I genuinely feel this might be the best idea I ever had.

After the invention of the horseless carriage, it supposedly took people years to realize that without reins to hold, the driver could now sit inside the cabin. Change of the core technology allowed a rethink of the entire product (the vehicle) but that rethink was hard.

With autonomous cars, we already have a rethink. Autonomous vehicles can be called to a person who needs it, so they should not be owned (and sit idle while the owner doesn't need to move) but should be taxis. Lots of people, includi... (Read more)

Somerville, like a lot of popular areas, has a problem that there are many more people who want houses than there are houses. In the scheme of things this is not a bad problem to have; mismatches in the other direction are probably worse. But it's still a major issue that is really hurting our community. I've been getting into a lot of discussions, and here are some ideas I find myself saying a lot:

• With the level of housing crisis we have right now I'm going to be in favor of basically any proposal that builds more bedrooms. Affordable housing, market rate housing, public housing,

In light of reading Hazard's Shortform Feed -- which I really enjoy -- based on Raemon's Shortform feed, I'm making my own. There be thoughts here. Hopefully, this will also get me posting more.

Consider the following program:

f(n):
    if n == 0:
        return 1
    return n * f(n-1)

Let’s think about the process by which this function is evaluated. We want to sketch out a causal DAG showing all of the intermediate calculations and the connections between them (feel free to pause reading and try this yourself).

Here’s what the causal DAG looks like:

Each dotted box corresponds to one call to the function f. The recursive call in f becomes a symmetry in the causal diagram: the DAG consists of an infinite sequence of copies of the same subcircuit.

More generally, we can represent any Tu