A: It’s complicated.
TL;DR: Two new well-designed studies find evidence that in-person schooling *does not* contribute to COVID spread when baseline community spread is LOW, but that in-person schooling *does* contribute to worsening COVID metrics when baseline COVID transmission is high.
Thus far the research on schools and COVID-19 has been quite muddy, but two new studies are helping to clear the waters. How can we make sense of the latest studies related to COVID-19 and schools?
Think through the three C’s: Comparison; Chance; and Context.
The recent reports on schools and COVID-19 have gotten quite a bit of attention. Today we’re partnering with Professor Chloe Gibbs (PhD, MA, MPP), an expert on education policy who compiles a weekly round-up of hot-off-the-press data about COVID-19 and schools.
We begin in classic Nerdy Girl style – by providing critical thinking tools to help you vet the evidence like a pro! So how does Dr. Gibbs sift through a quickly evolving science? Using a similar Three C’s approach that we’ve introduced in the context of (controversial) clinical trial data! (Editorial note: Remember the HCQ controversy – yikes.).
➡️ COMPARISON: How credible is the comparison between those exposed to the “treatment” and those who aren’t? In these studies, the “treatment” is in-person schooling, so we’re interested in whether treated places—those that reopened school—have experienced different COVID-related outcomes relative to places where schools remained virtual or were in a hybrid mode. Like trials for new vaccines, the best way to test cause and effect for new treatments is randomization. This ensures that the groups being compared are very similar in every away *except* for their mode of instruction. Of course, school districts are not too keen on being randomized to in-person or remote schooling, so we have to try the next best thing, sometimes called a “natural” experiment. We look for differences in exposure that are “as good as random” – for example places that are very similar in most ways but happen to differ in their school policies.
Two U.S.-based studies on schools and COVID-19 using this strong research design were recently released by education policy research centers, CALDER and REACH. In both studies, the researchers tried to compare apples to apples by comparing counties with similar characteristics and prior COVID trends but different choices regarding in-person schooling vs. hybrid or all-virtual instruction. Imagine comparing COVID trajectories in the “Twin Cities” Minneapolis and St. Paul if one city chose in-person and the other remote schooling. These statistical analyses to try to create “twin” counterfactuals for each location in the dataset.
The recent CDC MMWR report in contrast only described COVID-19 cases over time by age group, without testing whether those patterns were directly to school reopening policies. This type of description can be informative for seeing whether kids have high or low rates in general, but it doesn’t tell us much about cause and effect.
➡️ CHANCE: How likely were the results to have arisen just by chance? We want to know that what we observe in data represents reality, rather than statistical noise. Best protection: large sample size—Looking at many school districts for which we know details about reopening compared to just a handful—better ensures that the we’re not just picking up a statistical fluke (such as a particularly COVID-unlucky school district).
These studies have large samples of school districts with data on school reopening policies – the CALDER Center study with Michigan and Washington State data, and the REACH study with data on the vast majority of U.S. school districts.
The CDC MMWR analysis of COVID-19 incidence by age group has a large sample covering 44 states, DC, and three territories, but does not include any information on school reopening policies or instructional modalities.
➡️ CONTEXT: How well do the results translate beyond the research setting? To inform what we do, both in our individual decisions and in policymaking, we have to consider whether the findings of any particular study generalize beyond that specific study’s sample and setting. Best protection: replication across multiple geographies, time periods, and education systems. For example, when assessing the impact of different school reopening modes on COVID-19, it’s helpful to have data from lots of different places, both across the U.S. and around the world.
Previous studies were limited to Europe, mostly from the summer months, when case and school occupancy rates were low and mitigation measures like distancing and ventilation were facilitated by warmer weather. The U.S. based Calder Center and REACH studies demonstrate just how critical context is: both studies find evidence that in-person schooling does not contribute to COVID spread and health outcomes when baseline community spread is low, but that in-person schooling *does* contribute to worse COVID metrics when pre-existing COVID prevalence is high. The REACH study defined “high” as more than 36 to 44 new COVID-19 hospitalizations per 100,000 people per week in the county.
With COVID transmission high across much of the country, this new evidence is an important consideration in school closing/reopening policies. The REACH team provides a searchable database to gauge where your community stands relative to the problematic range of hospitalizations (link below). While the conclusions are less rosy than early evidence from the summer months in the U.S. and Europe, the findings are consistent with the fact that this early evidence came from contexts with low community spread.
➡️REMEMBER: *Science is a method, not a stable set of findings.* We should expect new and potentially changing guidance as scientists learn more. This is not suspicious, rather it’s a hallmark of the scientific method! We know that decisions about kids’ schooling, both the ones parents are making and the ones school administrators are making, are complex and involve consideration of difficult tradeoffs. These conversations are high stakes, heated, and characterized by uncertainty. We intend to present the latest evidence with transparency and humility, and hope it helps parents, policymakers, and school leaders as we all navigate these challenging times.
#school #COVID19 #thosenerdygirls Bump Club and Beyond Impact
LINKS:
Dr. Chloe Gibbs’s recent blog summarizing those two studies
The REACH Center study (and searchable database)
