Nice article all around!

Another error that conspiracy-theorists make is to "take the org chart literally".

CTists attribute superhuman powers to the CIA, etc., because they suppose that decision-making in these organizations runs exactly as shown on the chart. Each box, they suppose, takes in direction from above and distributes it below just as infallibly as the lines connecting the boxes are drawn on the chart.

If you read org charts literally, it looks like leaders at the top have complete control over everything that their underlings do. So of course the leader can just order the underlings not to defect or leak or baulk at tasks that seem beyond the pale!

This overly literal reading of the org chart obscures the fact that all these people are self-interested agents, perhaps with only a nominal loyalty to the structure depicted on the chart. But many CTists miss this, because they read the org chart as if it were a flowchart documenting the dependencies among subroutines in a computer program.

LW is academic philosophy, rebooted with better people than Plato as its Pater Patriae.

LW should not be comparing itself to Plato. It's trying to do something different. The best of what Plato did is, for the most part, orthogonal to what LW does.

You can take the LW worldview totally onboard and still learn a lot from Plato that will not in any way conflict with that worldview.

Or you may find Plato totally useless. But it won't be your adoption of the LW memeplex alone that determines which way you go.

Also, your empathy reassures them that you will be ready with truly helpful help if they do later want it.

I agree that a rich person won't tolerate disposable products where more durable versions are available. Durability is a desirable thing, and people who can afford it will pay for it when it's an option.

But imagine a world where washing machines cost as much as they do in our world, but all washing machines inevitably break down after a couple years. Durable machines just aren't available.

Then, in that world, you have to be wealthier to maintain your washing-machine-owning status. People who couldn't afford to repurchase a machine every couple of years would learn to do without. But people who could afford it would consider it an acceptable cost of living in the style to which they have become accustomed.

Did your really need to say that you'd be brief? Wasn't it enough to say that you'd omit needless words? :)

It seems unlikely that joining a specific elite is terminally valuable as such, except to ephemeral subagents that were built for instrumental reasons to pursue it.

It seems quite likely that people seek to join whatever elite they can as a means to some more fundamental ends. Those of us who aren't driven to join the elite are probably satisfying our hunger to pursue those more fundamental ends in other ways.

For example, people might seek elite status in part to win security against bad fortune or against powerful enemies. But it might seem to you that there are other ways to be more secure against these things. It might even seem that being elite would leave you more exposed to such dangers.

For example, if you think that the main danger is unaligned AI, then you won't think of elite status as a safe haven, so you'll be less motivated to seek it. You'll find that sense of security in doing something else that seems to address that danger better.

I've played lot of role-playing games back in my day and often people write all kinds of things as flavour text. And none of it is meant to be taken literally.

This line gave me an important insight into how you were thinking.

The creators were thinking of it as a community trust-building exercise. But you thought that it was intended to be a role-playing game. So, for you, "cooperate" meant "make the game interesting and entertaining for everyone." That paints the risk of taking the site down in a very different light.

And if there was a particular goal, instead of us being supposed to decide for ourselves what the goal was, then maybe it would have made sense to have been clear about it?

But the "role-playing game" glasses that you were wearing would have (understandably) made such a statement look like "flavor text".

Typo: "And that's why the thingies you multiply probabilities by—the thingies that you use to weight uncertain outcomes in your imagination,"

Here, "probabilities" should be "utilities".

Trying the pill still makes you the kind of person who tries pills. Not trying really does avoid that.