This is a special post for quick takes by Raemon. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
The 2018 Long Review (Notes and Current Plans)
I've spent much of the past couple years pushing features that help with the early stages of the intellectual-pipeline – things like shortform, and giving authors moderation tools that let them have the sort of conversation they want (which often is higher-context, and assuming a particular paradigm that the author is operating in)
Early stage ideas benefit from a brainstorming, playful, low-filter environment. I think an appropriate metaphor for those parts of LessWrong are "a couple people in a research department chatting about their ideas."
But longterm incentives and filters matter a lot as well. I've focused on the early stages because that's where the bottleneck seemed to be, but LessWrong is now at a place where I think we should start prioritizing the later stages of the pipeline – something more analogous to publishing papers, and eventually distilling them into textbooks.
So, here's the current draft of a plan that I've been discussing with other LW Team members:
— The Long Review Format —
Many LessWrong posts are more conceptual than empirical, and it's hard to tell immediately how useful they are. I think they benefit a lot from hindsight. So, once each year, we could reflect as a group about the best posts of the previous year*, and which them seem to have withstood the tests of time as something useful, true, and (possibly), something that should enter in the LessWrong longterm canon that people are expected to be familiar with.
Here's my current best guess for the format:
[note: I currently expect the entire process to be fully public, because it's not really possible for it to be completely private, and "half public" seems like the worst situation to me]
And importantly:
Next Year
Even if we stuck to the above plan, I'd see it as more of an experiment than the definitive, longterm review mechanism. I expect we'd iterate a lot the following year.
But one thing I'm particularly interested in is how this builds over the longterm: next year (November 2020), while people would mostly be nominating posts from 2019, there should also be a process for submitting posts for "re-review", if there's been something like a replication crisis, or if a research direction that seemed promising now seems less-so, that's something we can revisit.
Fair concerns. A few more thoughts:
First, small/simple update: I think the actual period of time for "canonization" to be on the table should be more like 5 years.
My intent was for canonization to be pretty rare, and in fact is mostly there to sort of set a new, higher standard that everyone can aspire to, which most LW posts don't currently meet. (You could make this part of a different process than a yearly review, but I think it's fairly costly to get everyone's attention at once for a project like this, and it makes more sense to have each yearly... (read more)
This is an experiment in short-form content on LW2.0. I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:
I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.