Both of my comments were about the thought experiment at the end of the post:
You are given a moral dilemma, either a million people will get an experience worth 100 utility points each, or a million + 1 people will get 99 utility points each. The first option gets you more utility total, but if we take the second option we get one more person served and nobody else can even tell the difference.
On average, road speeds went up by a whopping 16%!
But here’s something interesting:
Speeds on highways went up 13%, arterial road speeds went up by 10%, and local road speeds increased by 8%.
None of that’s 16%, and that’s important: This means congestion pricing sped roads up, but also sorted people to faster roads.
In response to having to pay a toll, people not only got off the road, they also made wiser choices about the types of roads they used!
This doesn't necessarily imply an improvement. If people switched to higher-speed but longer-distance roads, then total trip times could be similar (or more or less).
You start the thought experiment with this:
Lets say people can't actually tell the difference (in a one-shot trial) between experiencing 100 utility points of goodness, and only 99 utility points.
I'm assuming you don't mean there is literally no difference between the cases (which would make the answer obvious), but rather that people would be slightly happier in the Q100 case vs. the counterfactual Q99 case. They won't reliably be able to tell the difference, but there would be a small chance of any individual noticing the improvement. Still, if you multiply that epsilon by 1M trials, you get a noticeable effect.
I'm not sure if a barely-noticeable-difference in ice cream flavor is on the order of 1% of the total utility of a serving of ice cream, but I'm pretty confident that even if it was an order of magnitude less, you'd still be better off creating the 1M x 100 world than the (1M + 1) x (100 - ε) world.
nobody else can even tell the difference
Won't about 1% of them (10,000 people) be able to tell the difference?
Interesting. I think left/right arrows or triangles would be my most preferred/intuitive of the options.
Here's the chat fine-tune. I would not have expected such a dramatic difference. It's just a subtle difference in post-training; Llama 405b's hermes-ification didn't have nearly this much of an effect. I welcome any hypotheses people might have.
This looks like what happens when you turn the contrast way up in an image editor and then play with the brightness. Something behind the scenes is weighting the overall probabilities more toward land, and then there is a layer on top that increases the confidence/lowers variance.
The distinction in the quoted text seems backward to me since the 'x' in x-risk refers to 'existential', i.e. a risk that we no longer exist (extinction specifically), whereas 'doom' seems (to me) to merely imply getting stuck at a bad equilibrium.
Related: https://www.lesswrong.com/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment