This is how some of the most effective, transformative labs in the world have been organized, from Bell Labs to the MRC Laboratory of Molecular Biology.

I have a post currently in my drafts pile titled "Leading The Parade". Basic idea is that lots of supposedly-successful scientists, politicians, organizations, etc, were mostly "leading the parade" - walking somewhat in front of everyone else, but not actually counterfactually impacting the trajectory of progress much. It's the opposite of counterfactual impact.

On the other hand, at least some scientists etc were counterfactually impactful. But "how much people talk about how great they were" is not a very reliable proxy for how counterfactually impactful people/organizations were; we need to look at some specific kinds of historical evidence which people usually ignore.

I use two cases from Bell Labs as central examples:

When asking whether a historical scientist or inventor had much counterfactual impact or was just leading the parade, one main type of evidence to look for is simultaneous discovery. If multiple people independently made the discovery around roughly the same time, then counterfactual impact is clearly low; the discovery would have been made even without the parade-leader. On the other hand, if nobody else was even close to figuring it out, then that’s evidence of counterfactual impact.

For example, let’s consider two big breakthroughs made at Bell Labs in the late 1940’s: the transistor, and information theory.

In the case of the transistor, we don’t have evidence quite as clear-cut as simultaneous discovery, but we have evidence almost that clear-cut. First, the people who developed the transistor at Bell Labs (notably Bardeen, Brattain and Shockley) were themselves quite worried that semiconductor researchers elsewhere (e.g. Purdue) would beat them to the punch. So they themselves apparently did not expect their impact to be highly counterfactual (other than to who got the patent).

Second, there were actually two transistor inventions at Bell Labs. Bardeen and Brattain figured out their transistor design first. Shockley, unhappy at being scooped, came along and figured out a totally different design within a few months. It’s not quite simultaneous independent invention, but clearly the problem was relatively tractable and plenty of people were working on it.

So, the inventors of the transistor were mostly “leading the parade”; they didn’t have much counterfactual impact on the technology’s discovery.

Shannon’s invention of information theory is on the other end of the spectrum.

Ask an early twentieth century communications engineer, and they’d probably tell you that different kinds of signals need different hardware. Sure, you could send Morse over a telephone line, but it would be wildly inefficient. To suggest that any signal can be sent with basically-the-same throughput over a given transmission channel would sound crazy to such an engineer; it wasn’t even in the space of things which most technical people considered.

So when Shannon showed up with information theory, ~nobody saw it coming at all. There was (as far as I know) no simultaneous discovery, nor anyone even close to figuring out the core ideas (i.e. fungibility of channel capacity). Shannon was probably not leading the parade; his impact was highly counterfactual.

Summary and upshot of all this:

• It's pretty straightforward to read histories in order to look for counterfactual impact
• ... but hardly anyone actually does that.
• Standard narratives about which people/organizations were "highly impactful" seem pretty poorly correlated with counterfactual impact.
• So I expect there's a bunch of low-hanging intellectual fruit here, to be picked by going back through histories of lots of people/organizations and actually looking for evidence for/against counterfactual impact (like e.g. simultaneous discovery), and then compiling new lists of famous people/orgs which were/weren't counterfactually impactful.
• Absent that sort of project, I see claims like "This is how some of the most effective, transformative labs in the world have been organized, from Bell Labs to the MRC Laboratory of Molecular Biology", and I'm like... ummm, no, I'm not buying that these are some of the most effective, transformative labs in the world.

Organizational capture by sociopaths seems like just as bad a problem.

As a scientist I strongly agree. It seems like there's been a few steps towards this in recent years, for example with things like the Arc Institute or FROs. Hopefully this model gets the attention of government funders.

One of the theories on why more innovations (heliocentrism, discovery of the New World, etc.) happended in Europe instead of China (which is where astronomy, navigation, etc. were far more advanced) is that in China, there was lot more centralization. If you went to the Emperor and couldn't convince him that the earth was round, you were done. In Europe you could shop your idea around to a bunch of Kings and Queens until someone liked your idea.

Success in science and innovation isn't driven by efficiency. It's driven by diversity.  You want to have a lot of little groups trying out crazy ideas.

To understand this, it helpful to understand Moivre's equation, which he described in the article, “The Most Dangerous Equation.” I summarize it below.

Let’s say you wanted to figure out what causes kidney cancer. A reasonable question to ask might be, “which counties in the U.S. have the highest rates of kidney cancer?”

The answer is that rural, sparsely populated counties have the highest rates. You might think that perhaps this was due to pesticides, or lack of access to healthcare, or some other factor related to the rural lifestyle.

However, if you were to ask which counties have the lowest rates, you would find that rural, sparsely populated counties also have the lowest rates. In fact, the counties are often adjacent. See below. The red counties have the highest rates of kidney cancer, and the teal counties the lowest.

What’s going on?

Well, when you have only a few people in the county, the likelihood that there will be very high or low rates, due simply to chance, is high. For example, if there were only 2 people in the county and 1 person got cancer, that would be 50%. If 0 out 2 got cancer, it would be 0%.

This is why the best (and the worst) hospitals in the country or the best (and the worst) places to live often are small hospitals and small towns. And why the best (and the worst) science is done by small groups or individual scientists. Statistically, the smaller the sample size, the greater the likelihood of seeing an outlier.

This phenomenon was discovered by de Moivre, and made famous by Wainer’s.

The point is, as a group group gets larger and larger, innovation regresses more and more toward the mean. This is OK if you're working on a weakest link problem (where what you're doing is to prevent mistakes) but in endeavors like sciences, which is a strongest link problem (where what matters are the few huge breakthroughs), you want to make each group as small as possible.

I go into more detail in my post, https://medium.com/entrepreneur-s-handbook/why-the-most-dangerous-way-to-innovate-is-the-most-effective-way-dc9aad9b1b49

Thanks for posting this, I agree with the overall take that a block model is a superior alternative. I think some people in the Bay Area have idly looked into this for AIS funding; I was considering doing this myself but unfortunately had other obligations. 

This change would not get rid of the need for researchers to have a non-research skillset to secure funding.  It would just switch the required non-research skillset from 'wrangling money out of grant committees' to 'wrangling positions out of administrators'.  Your mileage may vary as to which of those two is less dysfunctional.

Bell labs, Xerox park, etc were AFAIK were mostly privately funded research labs that existed for decades and churned out patents that may as well have been money printers.  When AT&T (Bell Labs) was broken up, that research all but started the modern telecom and tech industry, which is now something like 20%+ of the stock market.  If you attribute even a tiny fraction of that to Bell Labs it's enough to fund another 1000 times over.

The missing piece arguably is executive teams with a 25 year vision instead of a 25 week vision, AND the institutional support to see it through; cost cutting is in fashion with investors too.  Private equity is in theory well positioned to repeat this elsewhere, but for reasons I don't entirely understand has become too short sighted and/or has significantly shortened horizons on returns.  IBM, Qualcom, TSMC, ASML, and Intel all seem to have research operations of that same near-legendary caliber, mostly privately funded (albeit treated as a national treasure of strategic importance); what they have in common of course, is they're all tech.  Semiconductor fabrication is extremely research intensive and world class R&D operations are table stakes just to survive to the next process node.

Maybe a good followup question is why hasn't this model spread outside of semiconductors and tech?  Is a functional monopoly a requirement for the model to work? (ASML has a functional monopoly on leading edge photo-lithography machines that power modern semiconductor fabs).  Do these labs ever start independently without a clear lineage to 100 billion+ dollar govt research initiatives?  Electronics and tech is probably many trillions in US govt funding since WWII once you include military R&D and contracts.

The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

New Zealand government research sort of does that. In 90s, the public research was reorganised into institutes - private companies but owned by government (government of day appoints the board members as position falls vacant). Initially, all government funding of these institutes was contestable - vaguely like the PI model. The cost and inefficiency of this system led to this be abandoned. Instead, the institutes get "Core" funding to support their core area (eg earth science in my case). Essentially the institutes propose very broad-brush programmes (so peer-driven) about how this will be spent. External panels (researchers from similar institutions overseas mostly, or possibly unis) critique it and a government bureaucracy evaluates against performance and alignment with government goals. Essentially it’s a negotiation process that sets a contract of around 5 years (from memory) which get tweaked as required. A similar model provides research funding (as opposed to teaching funding) to the universities. There are also several contestable funds, open to the whole research community. 

Does it work? I doubt any system is perfect and this one has a number of downsides. It is certainly a much more efficient system (less scientist-hours spent chasing money, and less bureaucracy evaluating bids) than what preceded it. There are advantages to the individual scientist though in pursuing contestable funding, though most will involve teams of scientists, often across multiple institutes/universities.  Core funding is tied to agreed programmes and the programme manager controls it. You can’t just do your thing on Core funding. Winning contestable funding means the team that won it is in control. Enough funding for your pet project and you can thumb your nose at institute management. 

The downsides as I see them: The % of government funding going to each institute is very static. It is hard to convince the government that your discipline is now hotter and more important than the discipline of another institute. The opposite was true of the contestable model - uncertainty of future funding made investment in research infrastructure (eg vessels) uncertain. Scrapping between institutes over money turned to scrapping within institutes over money. Perhaps this is actually a major upside - at least the scrappers know what they are talking about unlike the bureaucrats. The accountability model is everyone’s bugbear. It's public money so asking for accountability is obviously reasonable, but it generally feels like time-consuming paperwork to clueless bureaucrats. There must be a better way.

It should be noted that the institutes total funding thus consists of Core funding, contestable funding but also commercial research contracts. Depending on the Institute, this can be pretty high. Ours has been up 50% commercial. Others even higher.

Do you think CERN is an example of what you want?