How well did they follow the Wisdom of Crowds tenants? (James Surowiecki)
Diversity of opinion - Each person should have private information even if it's just an eccentric interpretation of the known facts.
Independence - People's opinions aren't determined by the opinions of those around them.
Decentralization - People can specialize and draw on local knowledge.
Aggregation - Some mechanism exists for turning private judgments into a collective decision.
I would say this is based on the Matt Ridley book The Rational Optimist? Rational Optimist Book | Matt Ridley
Brier score them?
What is considered the half-life of facts now?
“In an unlikely but conceivable turn of events, what if that scientist becomes infected with the virus, which leads to an outbreak and ultimately triggers a pandemic?” Fauci wrote in a paper reported on by The Australian. “Scientists working in this field might say — as indeed I have said — that the benefits of such experiments and the resulting knowledge outweigh the risks.” National Institute of Allergy and Infectious Diseases (NIAID) Director Anthony Fauci defended “gain-of-function” research in 2012
Flatland for machine learning?
How would you use a Brier score on this going forward? Also ran across this podcast last night on bias. https://knowledge.wharton.upenn.edu/article/want-better-forecasting-silence-the-noise/
Is there a trustworthy confusion matrix on for all the various testing methods? The drive to quote "cases" in a hysterical media seems to be misleading without it. Also a definition of "cases" seems to vary widely.
My other issue is a net deaths versus dyeing from other things previously. Is there a good metric seeing as there is a economic reason to label deaths for Covid over the other usual suspects.
Does any one believe that experts in this field are any better than others? Check out Tetlock's work on forecasting.
Tetlock decided to put expert political and economic predictions to the test. With the Cold War in full swing, he collected forecasts from 284 highly educated experts who averaged more than 12 years of experience in their specialties. To ensure that the predictions were concrete, experts had to give specific probabilities of future events. Tetlock had to collect enough predictions that he could separate lucky and unlucky streaks from true skill. The project lasted 20 years and comprised 82,361 probability estimates about the future.
The result: The experts were, by and large, horrific forecasters.... (read more)
“In aviation safety, tombstone mentality is a pervasive attitude of ignoring defects until people have died because of them.” - Wikipedia
For years, US aviation safety progress was on the backs of the dead for commercial carriers. To change at least pilot induced accidents, the major airlines internally created a program called the Advanced Qualification Program (AQP). This program wasn’t mandated by the FAA but developed by the carriers themselves. AQP in a nutshell is practicing the things that will kill you in a plane, not what the FAA tests for the ability to fly the plane. Most safety officials and insurance companies point to this as the single biggest factor in driving... (read 328 more words →)
How good would have been a pre-mortem as many products in tech get before they are released to the public? https://medium.com/paypal-tech/pre-mortem-technically-working-backwards-1724eafbba02