About a year ago, LessWrong held its second annual review, where we looked over the best posts from 2019. The LessWrong team offered $2000 in prizes for the top post authors, and (up to) $2000 in prizes for the best reviews of those posts.
Once again, some things came up and we got pretty sidetracked in actually getting around to awarding those prizes. Apologies to authors and reviewers. (I'm setting in motion some processes so that this year's prizes are given out in a much more timely fashion)
The $2000 in prizes for posts this year are awarded to:
The above authors are chosen primarily by the outcome of the vote (on which they all scored very highly). I'm also making a moderator call to award $100 to Benquo for Reason Isn't Magic, essentially coming out of our review-prize-budget. I thought it was a good review of the Secrets of Our Success post, which I think warranted special attention (see below).
Reviewing generally seems unglamorous compared to authoring posts. To compensate,I awarded more prize money here per unit-of-effort than I did for posts.
I particularly went out of my way to reward reviews that attempted a factual epistemic spot check – not because I think those are most important overall, but because they seem undersupplied to me, and more effortful. (AFAICT we only had 2-3?)
That said, much of LessWrong's content is in the domain of philosophy or theory, where epistemic spot checks aren't quite the right frame.
Secrets of Our Success
Book Review: Secrets of Our Success was an interesting case that stood out to me when reflecting on last year's process. Fiddler's review basically argues "This post mostly quotes cherry picked anecdotes, without distinguishing which of those anecdotes Scott endorses, and some of those cherry picked anecdotes don't even check out." No one rebuts Fiddler's review.
This seems worthy of investigation. I talked to some people about how they felt about Secrets of Success. A few people I respect said they agreed with some criticisms, and nonetheless voted for it because it contributed an important frame for looking at the world.
I'm still mulling over how I personally think about the concept. Meanwhile, it seemed valuable for our community reward signal to highlight critical discussion of the post, since it was rated as the #4 post in the review vote.
The awards for review prizes are:
If he weren't on the Lightcone team and ineligible for prizes, I'd be awarding jacobjacob a significant prize both for his many uses of the Elicit tool to crowdsource evaluation of claims during the Review. I also appreciated his epistemic spot check on Approval Extraction Advertised As Production – I disagreed with his method of evaluation, but liked the effort at an epistemic spot check, which I think was undersupplied in the Review. I also liked how jimrandomh came in with an alternate evaluation.
I'm super grateful to all the authors both for the original work that went into their posts, and their willingness to engage with a longterm feedback system, and to the reviewers for providing that feedback. There's plenty of improvement to build into our annual review system, but I'm enthusiastic about us continuing to build towards a robust intellectual ecosystem.
I'll be PMing you all shortly about how to claim your winnings.