At lunch we talked about hypothesis evaluation and generation, and I thought I'd put some of the thoughts I had into a post.
Here’s some points that seem relevant to me when thinking about institution building around this topic.
Regarding the current LW community, I see a few pieces of infrastructure missing before we're able to get all of this right.
Question: How does the best system we've ever built to solve this problem (edit: I mean science), deal with these two parts?
Thoughts on site-design that would help with this:
One significant open question that, writing this down I realise I have not got many explicit gears for, is how to have a process for updating the epistemic standards when someone figures out a way that they can be improved. Does anyone have any good examples from history / their own lives / elsewhere where this happened? Maybe a company that had an unusually good process for replacing the CEO?
Somewhat tangential, but...
You point to the following process:
Generation --> Evaluation --> Acceptance/rejection.
However, generation is often risky, and not everyone has the capacity to absorb that risk. For example, one might not have the exploratory space needed to pursue spontaneous 5-hour reading sprints while working full-time.
Hence, I think much of society looks like this:
Justification --> Evaluation --> Generation --> Evaluation --> Acceptance/rejection.
I think we some very important projects never happen because the people who have taken all the inferential steps necessary to understand them are not the same as those evaluating them, and so there's an information-asymmetry.
That's one of California's hidden advantages: the mild climate means there's lots of marginal space. In cold places that margin gets trimmed off. There's a sharper line between outside and inside, and only projects that are officially sanctioned — by organizations, or parents, or wives, or at least by oneself — get proper indoor space. That raises the activation energy for new ideas. You can't just tinker. You have to justify.
(This is one of the problems I model impact certificates as trying to solve.)
I think to some extent this is a chicken-and-egg problem. If there were already a venue for only polished accounts of ideas, with a scope that included mine, and if I posted there I'd be much more likely to get people like Wei Dai criticizing my stuff, I'd be much happier to post there. I think the way to do this might be to reach out on the individual level and coordinate a specific group of people, including people willing to serve as editorial staff, of something more like an academic journal.
To some extent the old Overcoming Bias was like an academic journal for the nascent rationality community, with Robin Hanson as editor. The comments weren't uniformly high quality, but the posts were great. However, it was still dual-purpose, serving both as a venue for high-quality exploration and things more like finished products. We need at least one of each, probably multiple exploration venues.
Slightly meta: I believe "Chicken-and-egg problem" is what we call a coordination problem with exactly 2 moving parts. (Or at least, that's how I was framing this problem internally when writing the OP.) And yeah, when you say 'reach out on the individual level', I think that's definitely a part of how to get it started - I think of it as adding free energy and doing things that don't scale, which is often the correct way to jump to a new Nash equilibrium. (Though I want to think carefully about the initial conditions before initiating, so that the new state unfolds well in the long run.)
On the object level, I feel like you underestimated the desired number of exploration venues by a couple orders of magnitude. Analogous perhaps to the ratio of math departments to math journals, say. Or subgroups-within-math-departments, to math journals.
This would be really cool. Perhaps the greatest danger is that such a journal might suck all the energy out of Less Wrong and/or the EA forum if it were to release articles too regularly?
Another consideration: common knowledge creation has a cooperative and competitive component. In a forum where anyone can post at any time, there's an implied "race to the bottom" accentuating the competitive aspect, as people worried the discourse will crystallize around a wrong idea may race to get their stuff in even half-baked, in order to head off something even worse. Standards to some extent help against this, but LW's current standards are decidedly mixed.