Other sources account for most of the variation in dance attendance,
and masking only plays a small part. The amount of the total variation
contributed by masking requirements is 5 %. This number is called the
coefficient of determination, and its square root is the correlation:
0.21. This correlation is low enough that we cannot conclude that
masking has a significant effect on attendance.
...
"But it still looks like masking has an effect!!" It does. It's just
that if the effect is there, it is small enough that we cannot
statistically prove an effect with just 44 dances. Assuming the
coefficient of determination really is 5 %, and we are aiming for a
traditional significance level of 0.05, the sample size curves tell us
we would need over 80 dances to be sure of the effect of masking.
The approach they took in their post involves some statistics that
make assumptions about the distribution of the data. While these
assumptions may well be right, now that we have fast computers we can
often use simulations to avoid this. I decided to have a go at
analyzing this data with a permutation
test.
In this approach, you permute (shuffle) the labels and then
check what fraction of permutations led to an outcome at least as
extreme as observed. In this case the attendance numbers were:
I decided to operationalize "at least this extreme" as the ratio of
the average attendance at mask required to mask optional dances and
wrote some code to simulate. With 10M
simulations, which coded in very lazy python run for 27s on my Mac, I
found that 19.3% of simulations passed that test, which can be
expressed as p=0.193.
The bottom line is the same, however: while the pattern we saw is more
likely in worlds where mask-optional dances are more popular, there's
enough variation from other sources that with 23 dances there's still
a decent chance that this apparent difference isn't real.
Entropic Thoughts recently reanalyzed the data I'd shared on the relationship between mask requirements and dance attendance we've seen at BIDA. They conclude:
The approach they took in their post involves some statistics that make assumptions about the distribution of the data. While these assumptions may well be right, now that we have fast computers we can often use simulations to avoid this. I decided to have a go at analyzing this data with a permutation test.
In this approach, you permute (shuffle) the labels and then check what fraction of permutations led to an outcome at least as extreme as observed. In this case the attendance numbers were:
I decided to operationalize "at least this extreme" as the ratio of the average attendance at mask required to mask optional dances and wrote some code to simulate. With 10M simulations, which coded in very lazy python run for 27s on my Mac, I found that 19.3% of simulations passed that test, which can be expressed as p=0.193.
The bottom line is the same, however: while the pattern we saw is more likely in worlds where mask-optional dances are more popular, there's enough variation from other sources that with 23 dances there's still a decent chance that this apparent difference isn't real.
Comment via: facebook, mastodon, bluesky