Re-reading Rationality From AI To Zombies
Reflections on Premium Poker Tools


Wanted: Foom-scared alignment research partner

I meet all six of those bullet points, basically. Emailed ya.

Believing vs understanding

Yeah that does make sense. I guess it depends on the feature in question and how close the competition is.

($1000 bounty) How effective are marginal vaccine doses against the covid delta variant?

I've thought about doing something similar with posing a question and offering a bounty for responses to the extent that they change my mind. I haven't pulled the trigger yet. I don't know why. Kudos to you for doing so!

Happy paths and the planning fallacy

In software development we sometimes play this game where you estimate how long a task will take, and then by sleight of hand the estimate becomes a commitment.

Yeah, I ran into that recently and it lead to issues. Not fun.

Preparing for ambition

I actually don't share that impression. The impression I get is that even for successful startups, there's a sort of hedonic adaptation where your sights end up being set on a new goal, which then produces stress. Eg. feeling content is fleeting and it is common to always feel perpetually "behind".

Preparing for ambition

To make sure I am understanding, are you saying that extra stress gives you an extra "oomph" that helps you achieve your goals? For me, it was the opposite, so marginal stress was harmful for me. Furthermore, my understanding is that this is true for others as well. Extra stress only provides an "oomph" when it is "good stress", not "bad stress".

Noticing Confusion

Seems worth referencing and linking to the HPMoR chapter Noticing Confusion.


$10M is a roughly standard value that the US government uses:

1000x, the short answer is "the singularity seems pretty plausible, and given how powerful a singularity would be, assuming we get the alignment problem right, I'd think we'd live really, really, really, long lives". I go into some more detail in, although that post is still very hand-wavy.


In the OP, jasoncrawford says:

We see this in the US with covid: The same people who didn’t realize that we all should be wearing masks, when they were life-saving, are now slow to realize/admit that we can stop wearing them.

At $10M/life, I can* agree that a mask isn't worth it. But if we eg. 1000x the value of life, wearing a mask provides something like $5 of value instead of $0.005, and at that price it would be worth it. So the question becomes whether we should value life wildly highly like this.

The point I was trying to make in my comment was, "perhaps we should value life wildly highly like this". The following is mentioned in the OP as a way to do "precognition": "2. Listen to other independent thinkers." As I explain in this post, I get the impression that if you go by the judgements of these expert, independent thinkers, it might be justifiable to value life wildly highly.

*Not wearing a mask feels like defecting in a prisoners dilemma to me, which makes me feel icky.

Load More