They are trying to explain the surprising fact that countries with high levels of mask wearing have correspondingly high "region spread" factors which cancel it out.
Their explanation is that this is because the regions most inherently susceptible to COVID-19 rationally respond by taking more protective measures (such as higher levels of mask wearing).
My point with the variance of the regional factor is that this makes it more likely that "region spread factor" is another term for "prediction error" rather than "inherent susceptibility".
I don't have specific knowledge about N95 quality masks. The Bangladesh RCT found that surgical masks were about 2x as effective as cloth masks (although that difference was on the edge of statistical significance). If I had to guess an equivalent RCT with N95s would find them to be ~2x as effective as surgical masks. But this post is mainly talking about masks = what average people wear and call masks.
The study was just released
For the cloth mask they got a 5% reduction in seroprevalence (equivalent to 15% for 100% increase) and for surgical masks they got an 9.3% reduction (equivalent to 28% for 100% increase).
I unequivocally lost the bet and will send my donation. Let me know if you have a preferred charity.
My current belief state is that cloth masks will reduce case load by ~15% and surgical masks by ~20%.
Without altering the bet I'm curious as to what your belief state is.
I can't accept the wording because the masking study is not directly measuring Rt. I would prefer this wording
"Gavin bets 100 USD to GiveWell, to Mike's 100 USD to GiveWell that the results of NCT04630054 will show a median reduction in cumulative cases > 15.0 % for the effect of a whole population wearing masks [in whatever venues the trial chose to study]."
I am also not convinced that zeroing out mask levels at the start solves this problem. The random walk variable is also a learned per region factor. Even if the starting value can no longer be influenced by mask levels, the starting value + 1 month of random walk value for the region can be influenced.
I'd like to respond to some of the points raised.
First I'd like to apologize for not reaching out to you before publishing my critique, I tried to integrate your responses from our email conversation but should have given you a chance to respond before publishing.
A minor point, for the data extrapolation you are reading the graph incorrectly. A higher growth difference meant that the growth rate (a rough approximation of R0) fell more sharply. The point of this section was not that the effect wasn't large, but that it pointed weakly in the wrong direction. Regions which increased mask wearing the most had their growth rates fall more slowly. I don't think this is strong evidence, but it does point against the effectiveness of mask wearing.
This is the spreadsheet used to compute the graph:
The core point of dispute if I understand it correctly is that knowing the absolute level of mask wearing in a region does not give evidence as to the overall R0 (even taking into account mobility and NPIs), but knowing the change in mask wearing over time gives evidence as to the change in R0.
In the model in your paper the absolute effect and relative effect are not disambiguated and an effect size mean of 25% reduction is observed.
In your proposed specification you try to isolate the relative effect by zeroing out starting mask wearing and observe a higher impact.
In my proposed specification I try to isolate the absolute effect by zeroing out the relative changes and observe a smaller impact.
These two observations don't contradict each other. The data is consistent with two distinct causal stories.
High inherent transmission -> Mask Wearing -> Lower transmission. This is your preferred model and indicates that the absolute effect is spurious.
High inherent transmission -> Mask Wearing + Lower Transmission later. This is my preferred model and indicates that the relative effect is spurious.
One reason to favor the second story is that although the model is described as measuring R_0, it is actually measuring Rt. The model does not include population infections as a modifier to the growth rate. This is a known causal factor which would artificially make masks look more effective.
As further evidence that our actual belief states are not too far apart, 15-25% reduction in case load (different from R0) was my best guess for the results of the mask RCT. I will make the $100 bet with the caveat being that payment is in the form of a donation to GiveWell.
I agree that this is a relative weakness of the model. I think part of it is that the division into vulnerable/invulnerable is a simplification. If for instance you injected somebody with COVID then everybody would be "vulnerable". So in some environments conditions are ideal for spread which makes many relatively immune people become infected.
I'm sympathetic to the case that education is signaling, but I think that case is less strong for early education. For instance this paper from Argentina uses teacher strikes to value a year of education at 6% of lifetime earnings.
That estimate is not wildly different and seems pretty immune to signaling.
Yeah, I don't think that HVAC in schools is something which will make a difference to their safety in time. My point was more
I would imagine that we could install experimental HVAC systems in a few hundred schools for not much money and get decent data.