Happy April!

...All I can think of is that it’s hard to imagine that just three weeks ago the world felt so different than it does today. I am still following COVID-19 a lot, so here’s my second semi-regular installment of a public consolidated brief that tries to consolidate everything I read into one short, actionable list so other people don’t have to re-create my work. This way I can save time and fight research debt.

Maybe read this instead of spending a ton of your own time obsessing? (Though do be wary that I am not an expert by any means and may be off in my selection and interpretation.)

I have a lot more reporting that I wanted to put in here but didn’t, due to lack of time. I will get them in the next issue and I will try to send out updates as fast as I can while maintaining a certain level of quality.

I do hope news will slow down at some point. ...I certainly will slow down at some point.


See also:

Doing Your Part! How You Can Stay Safe and Help the Fight!

The Wikipedia page “2019-2020 Coronavirus pandemic” is currently the second most viewed Wikipedia page of all pages this week, with hundreds of thousands of page views. If you like reading links, it could be really helpful to add a few minutes to your day to join me in this fight and keep this and the associated pages up to date. Also update the LessWrong links database and the Coronavirus Tech Handbook. And if you see parallel efforts, make sure they are aware of each other.


I don’t want to wade too much into the Great Masks Debate, which feels too complicated for me to adequately analyze and summarize at the speed at which I am writing this. I am not an expert, but since some are asking, I will nonetheless briefly summarize my provisional opinion:

Mask wearing is gaining steam. Austria already requires residents to wear masks while grocery shopping. And now a German city just mandated for the first time that people wear masks for shopping and using public transit.


Founders Pledge, in partnership with Silicon Valley Bank, have today launched a Covid-19 Response Fund.


Are you an engineer? New York is putting together the New York State COVID-19 Technology SWAT Team and is accepting volunteers.


If you can do epidemiological modeling and are not already crazy busy with COVID stuff, maybe you could now heed The Royal Society’s urgent call for epidemiological modeling.


My previous 29 Mar Brief has a bunch of advice on how to stay safe and contribute.

A Glance at The Latest Situation

Donald Trump backed off his original plan to re-open the country by Easter (insofar as he actually did have the power to do that) and is extending “social distancing guidelines” until April 30th. He also echoes claims made earlier by Dr. Fauci that around 200,000 people will die from COVID-19 by the end of the year.

While this is a very staggering and sad number, it is worth keeping in mind two things: (1) this number is only the modal part of the estimate… the actual death toll could be a fair bit higher or a fair bit lower and (2) this death estimate is still somewhat within the control of the United States government, should they be able to enact better policies to reduce the spread of COVID-19 and ensure hospitals are adequately resourced. Now is a good time to act.


There certainly are a lot of cases - with 952,171 worldwide cases as of the time of this writing, we look on pace to cross the 1M mark within a day.

Until recently, the most deadly infectious disease was tuberculosis, typically killing ~1.18M people per year, or 3231 people per day. Now COVID-19 has surpassed this daily death count and is the world's new most deadly infectious disease (Source).

A bit of good news though - while a bit too early to tell, it looks like we’re finally starting to see declines in new cases, especially in Italy:

We should get significantly more information in about two weeks or less about the shape of these curves. Projecting the curve forward suggests that things could be relatively stable by June.

Note that due to the log scale, the US is still in for quite a lot of pain.


Again, actual numbers being reported can be hard to compare and are complicated by differences in measurement. NYT reports a new source of discrepancies: “In Italy, authorities have conceded that their coronavirus death toll did not include those who had died at home or in nursing homes. Similarly in France, officials have said that only those who died in hospitals had been recorded as pandemic-related — a practice they said would change in the coming days.”


Lastly, all this stuff is just hard! The Atlantic reports:

We rely on numbers to understand the size and scope of tragedy—to gauge what went wrong and put the damage in perspective. More Americans have now died from the coronavirus than were killed in the September 11 terrorist attacks, multiple news outlets announced yesterday.
But we likely won’t have an estimate of how many Americans have died as a result of the pandemic for a very long time—maybe months, maybe a year. We will almost certainly never know the exact number. “It sounds like it could be totally obvious—just count body bags,” John Mutter, an environmental-science professor at Columbia University who studies the role of natural disasters in human well-being, told me in an interview this week. “It’s not obvious at all.”
When Hurricane Maria flattened Puerto Rico in September 2017, the storm’s devastation was overwhelming. Yet the official death toll in December stood at 64 people—a number that almost no one believed, as my colleague Vann Newkirk II wrote at the time. Nearly a year after the storm, a team of researchers tried to develop their own estimate. They gathered months’ worth of mortality data from households across the island and published a study concluding that, in actuality, more than 4,600 deaths were potentially attributable to the hurricane—70 times the official number.


Of course, yet another reason numbers may be hard to compare is just outright lying. After a lot of public suspicion, the US intelligence community is now reported as concluding that “China has concealed the extent of the coronavirus outbreak in its country, under-reporting both total cases and deaths it’s suffered from the disease” and that “China’s public reporting on cases and deaths is intentionally incomplete”.

The US also formally accuses Iran of lying about their numbers, saying, among other things that “[t]he regime is hiding a significant amount of information about the coronavirus outbreak. It is likely far worse that the regime is admitting. This lack of transparency poses a significant health risk to the Iranian people, as well as to Iran’s neighbors.”

If you’re confused by this - so are the US spies! Reuters reports “U.S. spies find coronavirus spread in China, North Korea, Russia hard to chart”.

...So Just How Bad Could This All Get?

Hungary descends into authoritarianism to fight the coronavirus:

Hungary's parliament has voted to allow Prime Minister Viktor Orban to rule by decree indefinitely, in order to combat the coronavirus pandemic, giving the populist leader extra powers to unilaterally enact a series of sweeping measures.
The bill, which has been criticized by international human rights watchdogs, has no specified end date and allows Orban to bypass a number of democratic institutions in his response to the outbreak.

This could be a cause for wider concern:

In Hungary, the prime minister can now rule by decree. In Britain, ministers have what a critic called “eye-watering” power to detain people and close borders. Israel’s prime minister has shut down courts and begun an intrusive surveillance of citizens. Chile has sent the military to public squares once occupied by protesters. Bolivia has postponed elections.
As the coronavirus pandemic brings the world to a juddering halt and anxious citizens demand action, leaders across the globe are invoking executive powers and seizing virtually dictatorial authority with scant resistance.
Governments and rights groups agree that these extraordinary times call for extraordinary measures. States need new powers to shut their borders, enforce quarantines and track infected people. Many of these actions are protected under international rules, constitutional lawyers say.
But critics say some governments are using the public health crisis as cover to seize new powers that have little to do with the outbreak, with few safeguards to ensure that their new authority will not be abused.

Gaze into the Crystal - The Latest Modeling and Forecasting

According to the Institute for Health Metrics and Evaluation (you may know them as the folks who put together the Global Burden of Disease report) the United States is potentially only two weeks away from peak hospital resource use (though this will vary significantly by region) and still faces a shortage of tens of thousands of hospital beds and ventilators.


COVID Act Now now provides county-level models for the US, showing time until projected hospital overload.


The crowdsourced prediction platform Metaculus has fancy new graphs displaying their predictions of cases, fatalities, and vaccine timelines:


So far experts have actually been underrating the spread of COVID even two weeks in the future… looks like they should’ve had wider error bars. Even a naive exponential trend extrapolation did much better than experts. Hopefully they will all learn their lesson and predict better next week:


Why is it so hard to make a good COVID-19 model? 538 reports:

Consider something as basic as data entry. Different countries and regions collect data in different ways. There’s no single spreadsheet everyone is filling out that can easily allow us to compare cases and deaths around the world. Even within the United States, doctors say we’re underreporting the total number of deaths due to COVID-19.
The same inconsistencies apply to who gets tested. Some countries are giving tests to anyone who wants one. Others are … not. That affects how much we can know about how many people have actually contracted COVID-19, versus how many people have tested positive.
And the virus itself is an unpredictable contagion, hurting some groups more than others — meaning that local demographics and health care access are going to be big determinants when it comes to the virus’ impact on communities

Now Let’s Talk Policy Response

Scott Gottlieb, the FDA commissioner from 2017-2019 and a current member of the Pence-led US government coronavirus taskforce, releases “National Coronavirus Response: A Roadmap for Re-Opening”. This is apparently the closest thing we currently have to a national strategic plan for the United States.

The plan operates in three broad phases, plus one bonus phase:

Phase I: Slow the Spread. Current phase of the response, where the US has widespread school closures, work-from-home, close malls and gyms, and limit restaurants. This is intended to stay in place until transmission has measurably slowed down and health infrastructure has scaled up.

Phase II: State-by-State Reopening. Individual states are able to move to Phase II as they are identified to be able to safely diagnose, treat, and isolate COVID-19 cases and contacts. Testing must be scaled up rapidly. These states can gradually reopen schools and businesses, but will likely need to maintain some degree of physical distancing and limitations on larger gatherings. Older adults will also need to remain at home.

The trigger for issuing a stay-at-home advisory in a US state is when case counts are doubling every three to five days (based on the current New York experience) or when state and local officials recommend it based on the local context (for example, growth on track to overwhelm the health system’s capacity). The trigger for issuing a recommendation to step down from a stay-at-home-advisory back to “slow the spread” is when the number of new cases reported in a state has declined steadily for 14 days (i.e., one incubation period) and the jurisdiction is able to test everyone seeking care for COVID-19 symptoms.

Phase III: Lifting restrictions. Phase III will also be gradually reached on a state-by-state basis “once a vaccine has been developed, has been tested for safety and efficacy, and receives FDA emergency use authorization” OR “there are other therapeutic options that can be used for preventive or treatment indications and that have a measurable impact on disease activity and can help rescue very sick patients”.

Phase IV: Rebuild Readiness for the Next Pandemic. Phase III is basically returning to normal, so that makes Phase IV more of a wishlist for what to do after things are normal to not get in this mess again. The report asks for greater vaccine development capacity to reduce time-to-vaccine below one year, improve the hospital surge capacity to be able to rapidly scale ICUs if needed in the future, improve disease forecasting, and move policy away from decentralized response.


Bill Gates: “Here’s how to make up for lost time on COVID-19”:

  1. consistent nationwide approach to shutting down
  2. step up on testing
  3. clear priorities for who is tested (prioritize health-care workers and first responders, followed by highly symptomatic people who are most at risk of becoming seriously ill and those who are likely to have been exposed)
  4. allocate masks and ventilators nationally, not by having each state compete
  5. don't speculate about treatments - run rapid trials involving various candidates and inform the public when the results are in
  6. start building the infrastructure to make vaccines now


Lawrence Summers summarizes the key policies: “I think we need to be investing, in a way far beyond what we are, in developing an infrastructure for widespread testing, widespread contact tracing, and widespread separation of those who are sick and those who are most vulnerable”


A bit more on China, misreporting, the Principal-Agent Problem, and Goodhart’s Law from the New York Times: “China Created a Fail-Safe System to Track Contagions. It Failed.”:

The alarm system was ready. Scarred by the SARS epidemic that erupted in 2002, China had created an infectious disease reporting system that officials said was world-class: fast, thorough and, just as important, immune from meddling.
Hospitals could input patients’ details into a computer and instantly notify government health authorities in Beijing, where officers are trained to spot and smother contagious outbreaks before they spread.
It didn’t work.
After doctors in Wuhan began treating clusters of patients stricken with a mysterious pneumonia in December, the reporting was supposed to have been automatic. Instead, hospitals deferred to local health officials who, over a political aversion to sharing bad news, withheld information about cases from the national reporting system — keeping Beijing in the dark and delaying the response.

China continues to have an issue of people not wanting to report bad news to higher-ups.


Another issue with creating good plans is when they aren’t followed at all. Politico reports that Trump team failed to follow the National Security Council’s pandemic playbook:

The NSC devised the guide — officially called the Playbook for Early Response to High-Consequence Emerging Infectious Disease Threats and Biological Incidents, but known colloquially as “the pandemic playbook” — across 2016. The project was driven by career civil servants as well as political appointees, aware that global leaders had initially fumbled their response to the 2014-2015 spread of Ebola and wanting to be sure that the next response to an epidemic was better handled.
The Trump administration was briefed on the playbook’s existence in 2017, said four former officials, but two cautioned that it never went through a full, National Security Council-led interagency process to be approved as Trump administration strategy. [...]
A health department spokesperson also said that the NSC playbook was not part of the current coronavirus strategy. [...]
Trump has claimed that his administration could not have foreseen the coronavirus pandemic, which has spread to all 50 states and more than 180 nations, sickening more than 460,000 people around the world. “Nobody ever expected a thing like this,” Trump said in a Fox News interview on Tuesday.
But Trump’s aides were told to expect a potential pandemic, ranging from a tabletop exercise that the outgoing Obama administration prepared for the president’s incoming aides to a “Crimson Contagion” scenario that health officials undertook just last year and modeled out potential risks of a global infectious disease threat. Trump’s deputies also have said that their coronavirus response relies on a federal playbook, specifically referring to a strategy laid out by the Centers for Disease Control.
It is not clear if the administration’s failure to follow the NSC playbook was the result of an oversight or a deliberate decision to follow a different course.“

Similarly, the US should have learned important lessons from the Ebola response that it seems to have not:

In international crises, policymakers and politicians rarely have a dress rehearsal before their debut on the main stage. Yet in retrospect, the Ebola outbreak of 2013–15 amounts to exactly that—a real-life test of Washington’s ability to detect and contain an infectious disease that threatens global security. [...]
It was clear to those who responded to the Ebola outbreak that the response system of the United States and the international response system would risk collapse if faced with a more dire scenario. It was equally clear that a more dire scenario taking place was a question of when, not if. [...]
Even before the Ebola epidemic ended, the U.S. government began pursuing a three-pronged strategy to contain a more dangerous outbreak. First, it doubled down on the Global Health Security Agenda, an initiative the Obama administration launched before the Ebola crisis to expand capabilities around the world to prevent, detect, and rapidly respond to infectious disease threats. [...] The strategy’s second prong was to further build out the network of hospitals and testing centers in the United States designated to treat Ebola and to increase the size of the national medical stockpile with more of the personal protective equipment and materials needed to fight highly lethal pathogens. The third prong was to designate a health emergency response coordinator and create a new Directorate for Global Health Security and Biodefense within the National Security Council. [...]
As 2017 turned to 2018 and 2018 turned to 2019, each prong of this strategy fell away like wheels off a bus. When the money provided by the Ebola Response Supplemental ran out, the new administration continued to fund the Global Health Security Agenda. But the overall budget for the Centers for Disease Control was cut, and no robust, new investments were made in greater deployable capability in the United States or other countries. At home, the envisioned expansion of the original 35-hospital Ebola Treatment Network did not take place; the $259 million appropriated for the network in 2014 was not followed by meaningful infusions of funds, setting it on track to expire in May 2020 and leading the Department of Health and Human Services to warn in November 2017 that “the current capacity of this system is not likely to be sufficient for many types of infectious disease outbreaks (e.g., pandemic influenza and other respiratory pathogens).” Nor was the national medical stockpile significantly bolstered. Congressional leaders passed budgets that had none of the vision or scale of the $5.4 billion Ebola Response Supplemental.
The third prong of the strategy was the last to go. In his first month as National Security Adviser, John Bolton shuttered the new NSC Directorate for Global Health Security and Biodefense. Its leader departed the NSC staff just one day after the WHO declared a new outbreak of Ebola in the Democratic Republic of the Congo that to date has killed over 5,000 people.”


We in the developed world should keep in mind that social distancing is a privilege that will be much harder to maintain in the developing world: “In neighboring Dharavi, Asia’s largest slum with a population of 1 million in an area of less than a square mile, workers laughed when I asked them if they had been practicing social distancing. As I stood outside a hovel, trying to keep some distance from people the day after Modi’s lockdown order was announced, even talking about social distancing felt obscene near a room of eight people crammed together with barely any space to breathe.”

Project Syndicate also reports “As horrific as this sounds, the situation in the advanced economies is likely to be much more benign than what developing countries are facing, not only in terms of the disease burden, but also in terms of the economic devastation they will face. [...] Under these conditions, even if developing countries want to flatten the curve, they will lack the capacity to do so. If people must choose between a 10% chance of dying if they go to work and assured starvation if they stay at home, they are bound to choose work.”

Notably, India and Pakistan have taken somewhat divergent approaches - India locked down with just four hours notice, whereas Pakistan has currently eschewed a countrywide lockdown, instead closing large malls and schools but letting individual states and provinces manage the rest.

COVID testing in India and Pakistan still remains too low to tell yet how each strategy is working.


IDIsnight provides some advice for developing countries:

  1. Lockdown as much as is possible
  2. Use painted or marked “social distancing squares” to cue people to stay distant from each other for stores that remain open
  3. Remind people to use frequent handwashing and cloth masks
  4. Avoid outright closing public transit, but close especially risky forms of transit
  5. Address issues of urban-rural migration
  6. Make sure social distancing policies are localized and contextualized
  7. Identify trusted leaders and communicate messages in many local languages

A Bit About Life Under Quarantine

You may have heard contestants on the “Big Brother” TV show were sequestered from news and were some of the last to learn about the pandemic. But they’re not the actual last. It turns out that many submariners are still unaware of the pandemic:

Mariners aboard ballistic submarines are habitually spared bad news while underwater to avoid undermining their morale, say current and former officers who served aboard France’s nuclear-armed subs. So any crews that left port before the virus spread around the globe are likely being kept in the dark about the extent of the rapidly unfurling crisis by their commanders until their return, they say. [...]
Speaking exclusively to The Associated Press, Salles said he believes submariners will likely only be told of the pandemic as they head back to port, in the final two days of their mission. [...]
“No matter how serious an event is, there is nothing a submariner can do about it. And since he cannot do anything, better that he know nothing,” Salles said. “They know that they won’t know and accept it. It’s part of our deal.”


Buzfeed reports: “The coronavirus outbreak has prompted an unprecedented surge in gun sales, exceeding a previous record set after the Sandy Hook mass shooting.”


“It Felt Like a Black Mirror Episode”: The Inside Account of How Bird Laid off 406 People in Two Minutes via a Zoom Webinar


Deals for businesses in the time of COVID-19… now’s a great time to get some free software!

If You Still Own Envelopes, Check Their Backs - Here’s the Latest Cost-Benefit Analysis

Another paper finds net benefits to lockdown:

We examine the net benefits of social distancing to slow the spread of COVID-19 in the United States. Social distancing saves lives but imposes large costs on society due to reduced economic activity. We use an SIR model to perform a benefit-cost analysis of controlling the COVID-19 outbreak. Assuming that social distancing measures can substantially reduce contacts among individuals, we find net benefits of roughly $5 trillion in our benchmark scenario. We examine the magnitude of the critical parameters that would lead to negative net benefits. A key unknown factor is the time to economic recovery with and without social distancing measures in place. Our sensitivity analysis points to a need for effective economic stimulus when the outbreak has passed.


Also another paper suggests locking down somewhere between 7 to 34 weeks:

We investigate the optimal duration of the COVID-19 suppression policy. We find that absent extensive suppression measures, the economic cost of the virus will total over $9 trillion, which represents 43% of annual GDP. The optimal duration of the suppression policy crucially depends on the policy’s effectiveness in reducing the rate of the virus transmission. We use three different assumptions for the suppression policy effectiveness, measured by the R0 that it can achieve (R0 indicates the number of people an infected person infects on average at the start of the outbreak). Using the assumption that the suppression policy can achieve R0 = 1, we assess that it should be kept in place between 30 and 34 weeks. If suppression can achieve a lower R0 = 0.7, the policy should be in place between 11 and 12 weeks. Finally, for the most optimistic assumption that the suppression policy can achieve an even lower R0 of 0.5, we estimate that it should last between seven and eight weeks. We further show that stopping the suppression policy before six weeks does not produce any meaningful improvements in the pandemic outcome.

Now Just What are the Tech Overlords up to?

A new COVID-19 High Performance Computing Consortium brings together the US government, IBM, Amazon, Google, Microsoft, Hewlett Packard, MIT, NASA, and others to “provide COVID-19 researchers worldwide with access to the world’s most powerful high performance computing resources that can significantly advance the pace of scientific discovery in the fight to stop the virus”.


Palantir provides COVID-19 tracking software to CDC and NHS, pitches European health agencies… but it will probably be ok?

Palantir, a secretive government-friendly big data operation that’s able to ingest vast amounts of information to visualize trends and track individuals — useful tasks as the spread of COVID-19 threatens to overwhelm healthcare systems and ravage economies.
In mid-March, The Wall Street Journal reported that Palantir was working with the CDC to model the potential spread of the virus. Forbes reports that CDC staffers are now regularly using Palantir’s web app to visualize the spread of the virus and to anticipate hospital needs. According to that report, Palantir is eschewing dealing with sensitive personally identifying information in its coronavirus efforts, instead providing analysis of anonymized hospital and healthcare data, lab results and equipment supplies through a platform called Palantir Foundry.
[...] Likely aware of its reputation as the shadowy tech giant that helps to power ICE’s deportation machine, Palantir is apparently acknowledging the privacy implications of its new work. In a statement provided to The Wall Street Journal, Palantir’s privacy lead Courtney Bowman asserted that privacy and civil liberty must be taken as “guiding concentrations” in any data-driven COVID-19 response, “not as afterthoughts.”
While it appears to be taking on a new role with the U.S. COVID-19 response, Palantir has worked with the U.S. federal government on infectious health threats for years. In 2010, the CDC used Palantir to monitor an outbreak of cholera in Haiti.”


Amazon begins running temperature checks and will provide surgical masks at warehouses.

And How Do We Get Out of this Mess? Vaccines, Treatments, Testing, Tracing, etc.

Johnson & Johnson announces a vaccine that could be potentially available by early 2021:

J&J today announced the selection of a lead COVID-19 vaccine candidate from constructs it has been working on since January 2020; the significant expansion of the existing partnership between the Janssen Pharmaceutical Companies of Johnson & Johnson and the Biomedical Advanced Research and Development Authority (BARDA); and the rapid scaling of the Company’s manufacturing capacity with the goal of providing global supply of more than one billion doses of a vaccine. The Company expects to initiate human clinical studies of its lead vaccine candidate at the latest by September 2020 and anticipates the first batches of a COVID-19 vaccine could be available for emergency use authorization in early 2021, a substantially accelerated timeframe in comparison to the typical vaccine development process.

J&J has also committed to produce more than a billion doses for global use, including production in both the United States and overseas.


FDA Emergency Use Authorization is granted to a new system for decontaminating N95 respirators for re-use. Innovations in this area could help with the mask shortage.


Does hydroxychloroquine help? From Derek Lowe: there are now two blinded, randomized, and controlled trials from china. Unfortunately, “these two came out rather differently, with the Zhejiang study showing no detectable difference on treatment and the Wuhan one showing what looks like a real effect[...] which one (if either) reflects the real-world situation?”

538 also urges some patience and caution:

No drugs currently on the market have previously gone through control trials to treat COVID-19 specifically. Because of this, we can’t be certain how effective or safe they would be. And even when a treatment seems promising, it may not end up being effective: Lopinavir-ritonavir, a combination of anti-HIV drugs, was considered a possible treatment for COVID-19, but a clinical study published March 18 showed the pair had no substantial benefit on patients.
Other clinical trials for existing medications, such as remdesivir, have already begun, including one sponsored by the National Institutes of Health and led by Kalil. It will take weeks, and possibly months, for the trials to be completed, and there’s a chance that none of the drugs being investigated will effectively treat COVID-19. But it’s the only hope we have of figuring out whether these drugs actually work and are safe to use.

The Non-Profit Impacts

Run a US 501c3 non-profit organization with less than 500 employees? Want grant money from the US government if you don’t lay off employees? Reach out to your retail banking partner and tell them “I want to apply for the CARES ACT SBA LOAN 7(a)”. Details here. Act quickly!

Don’t Forget About the Nonhumans!

Cynthia Schuck and Wladimir Alonso, two public health experts who are also passionate about the plight of farm animals, have prepared a short e-book to provide factual information on the connection between farm animals and pandemics.

Your Regular Dose of WTF

Guess who else is quarantined?


Fun (Online) Distractions, Because We All Still Need to Enjoy Life

Michael “The Situation” Sorrentino promotes “social distancing”.

Super Mario remasters coming to Nintendo Switch for 35th anniversary.


Thanks to countless people for helping me find all the links in this list!

New to LessWrong?

New Comment
9 comments, sorted by Click to highlight new comments since: Today at 4:49 PM

According to my calculations from https://covidtracking.com/data/us-daily, the number of tests per day in the US has stayed flat since a week ago (97,806 on Mar 26, 100,989 on Apr 1), while the positive rate has gone from 17% to 26%. If more tests were being done to keep pace with actual infections, the curve for the US wouldn't be bending so much.

Something similar may or may not be happening elsewhere, but we can't really trust "projecting the curve forward" without looking into more details.

[EDIT: the following is mistaken and the claim in OP was correct, though that wasn't knowable from the publicly released data. See habryka's comment.]

Many of the expert predictions were indeed crazily optimistic and had tiny error bars, but there's a problem with the story. FiveThirtyEight mistakenly reported (and they still haven't updated this!) that the March 16-17 survey asked experts about the number of cases reported on Covid Tracker on March 29, when in fact the study asked about March 23rd.

The correct number on the 23rd was 42,152. This was of course in line with the exponential extrapolation, and it was worse than the worst-case estimates of 13 out of 18 researchers, but at least their estimates show only typical levels of insanity and incompetence.

I think this is wrong. I've heard of multiple people who have reached out to the authors and illustrators for the article, who have said that the data is indeed correct, but wasn't published in the survey. Here is the relevant tweet response: 


Nice legwork! It's insanity and incompetence on the part of the experts after all.

Metaculus (me included) also did similarly poorly on the question of US case growth. Out of all Metaculus questions, this one was probably the one the community did worst on. Technically expert epidemiologists should know better than the hobbyists on Metaculus, but maybe it's a bit unfair to rate expert competence based on that question in isolation.

What was surprising about it was mostly the testing ramp-up. The numbers were dominated by how much NY managed to increase their testing. I managed to overestimate the number of diagnosed cases in the Bay area, while still heavily underestimating the number of total cases in the US.

This is the relevant Metaculus question: https://www.metaculus.com/questions/3712/how-many-total-confirmed-cases-of-novel-coronavirus-will-be-reported-in-the-who-region-of-the-americas-by-march-27/

If you look at the community median at a similar date to the prediction by expert epidemiologists, it's also off by a factor of 6 or so. (Not sure what the confidence intervals were, but most likely most people got negative points from early predictions.)

(For those interested, the Metaculus user "Jotto" collected more examples to compare Metaculus to expert forecasters. I think he might write a post about it or at least share thoughts in a Gdoc with people who would be interested.)

What would you expect to happen if those experts started participating in Metaculus?

I mostly made my comment to point out that the particular question that's being used as evidence for expert incompetence may have been unusually difficult to get right. So I don't want to appear as though I'm confidently claiming that experts need a lesson on forecasting.

That said, I think some people would indeed become a bit better calibrated and we'd see wider confidence intervals from them in the future.

I think the main people who would do well to join Metaculus are people like Ioannidis or the Oxford CEBM people who sling out these unreasonably low IFR estimates. If you're predicting all kinds of things about this virus 24/7 you'll realize eventually that reality is not consistent with "this is at most mildly worse than the flu."

Or an error in the editorial process that for some reason people are doubling down on. I do think that's a serious option.


This puts a new light on experts getting the predictions wrong. People are speculating that some of the California cases date back to January or even December. Similar stuff could have happened in New York. IMO, that's the type of thing that makes sense to have outside one's 95% confidence interval.

EDIT: OTOH it seems as though the infections only started in New York in February, and yet they spread to infect a large portion of the population there (tentative serology estimates say about 20% for the city). It doesn't seem to be the case that the wide spread is explained by the infection in New York having started a lot earlier than expected. But something about this confuses me. If the infections reached the Bay area months earlier than they reached in New York, why is New York worse off? I guess one unusually thing about New York is how insanely little space they have inside restaurants and so on. Go to a California Starbucks and it's awesome and comfortable. Go to a New York Starbucks (wasn't it even invented there??) and you can't even sit anywhere and there are walls all around you. Probably infections just spread way faster in that tightly crammed setting?