Link to study in the Proceedings of the National Academy of Sciences
A: Based on all of the available evidence, we are wearing our masks and strongly encourage others to do so. Sadly, this study is a HOT MESS…
How can this be? The *Proceedings of the National Academy of Sciences,* that’s a serious journal, right? Yes- AND No. PNAS is a reputable journal, but this never guarantees the quality of published papers. We’ve seen recent COVID-19 papers from another top journal, The Lancet, retracted after the data could not be accessed for replication.
The REAL peer review happens when the broader scholarly community reads and scrutinizes a paper’s methods and findings. Fortunately, this is happening at in real time for COVID-19 research, especially for papers garnering big headlines.
So what’s up with this PNAS paper?
For an excellent blow-by-blow, we recommend this insightful twitter thread by Noah Haber.
One choice quote: “It’s hard to write a summary critique of this paper, because it commits … all of them, plus some creative ones for fun. You’ve got ecological fallacy, inappropriate projections, NO treatment of statistical uncertainty (!!!!), attributed causality, non-generalizability, etc.”
As several of the Nerdy Girls are epidemiologists, we will highlight two of the biggest shortcomings:
1) The paper (written by chemists) does not use even a simple epidemiological model of transmission but plots transmission as a linear prediction (a straight line). This is not realistic representation of infection spread in the least.
2) Using case counts in 3 locations (Wuhan, New York City, and Italy), the authors then attribute deviations from this straight line as caused by one change in policy (mandatory face coverings).
It doesn’t take a statistician to see that 3 locations are not a large sample, and that MANY other things were changing over this time period—like almost EVERYTHING about daily life in all of these places, in fact.
Not to mention in real life we would expect a lag of at least a couple of weeks between any dramatic policy changes and changes in infections.
We could go on. And on. Some have called for the paper to be retracted it is so shoddy. So why are we calling attention to a paper that supports the efficacy of masking but is actually a scientific dumpster fire?
COVID-19 research is coming out fast and furious, and mostly this is a good thing for quickly building a crucial knowledge base. But like in any line of work, quality and rigor do vary.
In a time when misinformation is truly a matter of life and death, we demand high standards and scrutiny for all science, not just those whose findings match what we already believe to be true!
This is crucial for scientific credibility in the long run. And in the long run, we all want the same thing–CORRECT answers for how best to minimize COVID-19 risk.
Wear those masks! But read this research for evidence:
