Elicit prediction for the probability that there will be more than 5.5 submissions that receive a payout by October 13th (the title doesn't mention receiving a payout, but I intended for it to).
My thinking: a quick scan of the Book Reviews tag indicates about 2.5 posts with that tag per month. I suppose another 0.5-1 or so are book reviews but just haven't been tagged as such. So that is my baseline. From there, I expect the $500 reward to give a solid bump, but nothing too crazy.
Based on the Book Reviews tag scan, this has been a smashing success. My scan around lunch showed ~33 entries which are "1 month" or younger. However, I don't know how many will get payouts.
I will have to count. I've been paying out for the ones that meet the bar (quite a few! most?) and who have requested payment. I'll say this again in a public post, but if you've written a book review and would like to claim the bounty, please message me on Intercom.
I just found out that Pinker is releasing a book called "Rationality" (Hmm.. sounds familiar..) later this month, which apparently presents tools that "have never been presented clearly and entertainingly in a single book--until now." (Good that someone finally put in the effort!).
But slight sarcasm aside, it seems like the sort of thing our community should keep up with - so if someone was looking for a book to review, this sounds like a good option (though it doesn't give much time, it releases on the 28th this month and the bounty closes on the 14h of the next).
I'm already working on a different book review (which is long and difficult to write) so I won't be doing this myself.
Good idea, I might give this a shot. I hope others give it a shot regardless of whether I do since I want to read others’ reviews.
This seems really good! As a specific offer, and as part of a general class of LW initiatives.
Presumably most people who would take you up on it already trust LW, and wouldn't be too worried about the open-ended nature of the "if we like it". But I hope you can make that somewhat more concrete for people who do opt to contact you with a proposal / to seek feedback. It would be nice to be able to give assurances of the form "if you do X and Y, and the review has quality similar to your past posts, and length around Z, then we will pay for sure."
It's possible that after doing the first round, we'll be a better position to clarify what makes for a review that we like, though I do expect it to be hard to express intensionally.
Well gang, today's the last day. A rough count from a few minutes ago, judging by the Book Reviews tag, put us at ~31 potential entries, and the day isn't over yet.
Congratulations on the huge success! I do not envy you the judging workload thus generated.
Sounds like a great idea, and like something you might want to ask Scott to publicize on the next ACX Open Thread.
I estimate that our best blog posts generate much more than $500 of value.
Just curious - how are you estimating value here? I’m totally excited for this policy, just wondering how you put a dollar amount on this.
I have to confess, it is a more personal intuition (and willingness to spend altruistic dollars) than a hard calculation. It has to be that way, because at some point I have to assign dollar value to something that isn't a dollar value.
A piece of it is that I think individuals (and collectively the community) understanding the world a bit better is worth quite a lot. I think that knowledge has compounding gains, so each additional piece that is known gets built upon and multiplied. I think that our community becomes stronger and wise as a result of scholarship. And of course, each good piece of writing strengthens the community, gets more readership, and in turn generates more writing. This is a poor articulation, but it's what I can manage in a spare couple of minutes.
Another way to think about it is the labor costs. Generating posts is difficult knowledge, the likes of which you'd pay at least $50/hour in the Bay Area. A solidly written post might take between 5 and 20 hours, which means if you were pay someone an hourly wage for the post, the amount that it'd make worth it, in dollars, is like $500. If someone is going to bother writing that post, really there ought to be some "profit" with the post being worth more.
Does this apply to review for books that have already been reviewed on LW? I would assume that in this case you want a different approach for the review, but it's not clear whether it's valid or not.
"Conditions
...
5. You may review a book that was already reviewed on LessWrong (or SlateStarCodex/ACX), however your review must add significant value beyond the existing review(s)."
- ...sometimes I worry that rationalists are too interested in thinking about the world by introspection or weird analogies relative to learning many facts about different aspects of the world; I think book reviews would maybe be a healthier way to direct energy towards intellectual development.
Data is the foundation of empiricism. Abstract reasoning untethered to factual reality (or mathematical axioms) is not rationality.
Slightly, but only slightly off-topic:
When I change to a different sorting there, the "load more" link disappears.
This is a bug when you change sort method after having clicked load more once. Try refreshing.
You can vote for a specific tag on a page, and "most relevant" sorts by the post where that tag has been upvoted the most instead of the karma of the posts with the tag.
Your book review must be published after the posting of this announcement, i.e., no submitting book reviews you wrote a month ago and already published elsewhere on the Internet.
What's your policy on previously-partially-published reviews? The specific case I have in mind is a rough review I put up on Goodreads, which would need major reworking to be suitable here. (It's currently more of a notes-dump than a proper review.)
"Karma is a strong correlate of quality (whether or not the bounty is paid out is not strictly contingent on the karma it gets, but is influenced by it).
Importantly, quality is not the automatic result of effort. Someone could expend a lot of effort writing an extremely long and detailed review that no one wants to read because it’s tedious or because the English is grating. "
Before anyone gets sad:
While Karma is certainly a useful measure for the probability that a book review will be seen as rewardworthy by the LW team, nobody really knows how strongly it correlates with "quality" as defined by what non-LW readers would see as high-quality. Saying that "Karma is a strong correlate of quality" is not an objective description, but a belief.
Unreadable book reviews would probably be seen as having low quality both on LW and on goodreads or other review sites. Readability is only one factor leading to more Karma points, however, and I assume that obvious things like topical fit with things the typical LW reader likes and less obvious things like being close to the center of the social network of LW will lead to higher Karma points. Therefore, we could say that Karma measures quality in the same sense that IQ measures intelligence: It would then just measure what it's defined to measure, so the word "quality" could just as well stand for "Karma points" without mixing words and associations. The correlation betwee Karma points and quality in this sense then is 1.
Kudos to Kelsey Piper and Buck for this idea. See Buck’s shortform post for another formulation.
LessWrong is trialing a new pilot program: paying USD500 for high-quality book reviews that are of general interest to LessWrong readers, subject to our judgment and discretion.
The program will by default run for one month (until October 13). At the end of the month, a bonus $750 will be split evenly between the top three book reviews received, as judged by us.
Most non-fiction topics related to science, history, and rationality will merit payment if the book review is of sufficient quality. By “quality” I’m referring to both content and form. Do the inferences seem correct? Does the reviewer seem to be asking the right questions? Does the summary feel informative or lacking? Do I feel confused or enlightened? Is it riveting or a slog to get through? On the writing side, relevant aspects are sentence construction, word choice, pacing, structure, imagery, etc.
I don’t want to be too prescriptive about form since I expect that being of sufficiently high quality (nebulously defined) is enough to make for exceptions, but generally, I’m interested in book reviews that:
(An extra great format is to compare and contrast two or more books on the same topic.)
Since it’s hard to give an explicit definition of “quality”, I’m going to fall back on examples and hope that these are better than nothing. Generally, the book reviews tag is a good guide to the kinds of book reviews that are popular on LessWrong and that we want to incentivize.
Below I’ve listed specific book reviews that were either particularly great or kind of poor. Again, most of these came down to quality rather than topic.
These book reviews all present engagingly on a topic of interest. They’re not difficult to read, and having read them, I know something more about the world than I did before.
I am reluctant to name and shame particular essays on LessWrong, and instead, direct people to view the book reviews tag sorted by karma and look at the lowest scoring posts (you’ll have to click load more to get the entire list). Karma is a strong correlate of quality (whether or not the bounty is paid out is not strictly contingent on the karma it gets, but is influenced by it).
Importantly, quality is not the automatic result of effort. Someone could expend a lot of effort writing an extremely long and detailed review that no one wants to read because it’s tedious or because the English is grating. To be explicit, the bounty will not be paid out just because someone put a lot of effort into their review.
However, to make it easier to produce high-quality reviews, anyone writing a book review for this program is welcome to avail themselves of LessWrong’s feedback service, even if they don’t yet have 100+ karma. Just ping us on Intercom.
Foremost, we want more valuable content on the site. We are beginning to experiment with offering people monetary compensation for their hard work. I estimate that our best blog posts generate much more than $500 of value. However, $500 is maybe enough to symbolically thank our writers and incentivize them.
Beyond the first-order benefit, there could be additional benefits, as listed by Buck:
- It might encourage people to practice useful skills, like writing, quickly learning about new topics, and thinking through what topics would be useful to know more about.
- ...sometimes I worry that rationalists are too interested in thinking about the world by introspection or weird analogies relative to learning many facts about different aspects of the world; I think book reviews would maybe be a healthier way to direct energy towards intellectual development.
- It might surface some talented writers and thinkers who weren’t otherwise known to EA [or LessWrong]
Of these, I’m especially interested in helping to develop new strong writers and researchers.
We’re starting with compensation for book reviews as these feel like a more “approachable” kind of content for people to target writing. By being a more specific format, I imagine that it will be easier for people to get started than if the directive were “write good posts”.
This list will be expanded as things come up that I didn’t think of.