A pet peeve of mine has always been how people decide to give ratings on Yelp. If the restaurant is pretty good, it gets five stars. If it's ok, it gets four stars. If it's genuinely bad, it gets three. If the waiter was rude, it gets one.

This leads to a situation where most ratings range from 4 to 5. A 4.8 is a good rating. A 4.2 is underwhelming.

I think this is something that we've all internalized. In some sort of a priori sense, you'd think that an average of 4 out of 5 stars is pretty good. However, we all know that it isn't. In fact, this extends beyond Yelp. I remember hearing that Uber drivers get fired if their rating drops below 4.7 or something.

This works because deep down, we're all Bayesians. We don't take things literally. If we did take things literally, we'd see a place with an average of 4.2 stars and expect good things. After all, Yelp labels a 5 star review as "Great", 4 stars as "Good", 3 as "Ok", 2 as "Could've been better", and 1 as "Not good".

Instead of taking things literally, we look at the rating as Bayesian evidence. We realize that a 4.2 rating isn't something that we'd expect to observe if the restaurant actually is really good, and it is something we'd expect to observe if the restaurant is mediocre.

So then, maybe there's no problem here. Maybe things all work out at the end of the day. The fact that raters lean so heavily towards giving ratings of 4 and 5 doesn't actually prevent users from using the average rating to tell how good a restaurant is. Users just need to re-calibrate.

It's the same thing as that friend who is overenthusiastic in their text messages. Everything has a bunch of exclamation points and emojis. And so, when you receive a text saying "good to hear from you", you're concerned. If they actually were happy to hear from you, you'd expect something more like "HEY!!! So good to hear from you!!!!!!!". If they were just normal-pleased to hear from you you'd expect "Hey! Great to hear from you!!!".

I think in most contexts this sort of skew towards positivity is fine. People are able to read well enough between the lines. Recalibrating would take more effort, require too much coordination, and cause too much social friction for it to be worth it. However, I can imagine contexts where accurate feedback is essential, and it would be worth the effort to incorporate 1 and 2 star reviews, so to speak.

Such contexts are probably rare. The participants would have to really care about being effective. Off the top of my head, a few candidates that come to mind are alignment researchers, cofounders of a startup, and a mentee working with a mentor. In situations like office jobs where people only pretend to care about being effective, pursuing such a culture would probably just backfire and cause friction.