As some states moved to open their economies from the pandemic lockdown this week, The New York Times reported that a government model predicted a dramatic increase in the death toll from COVID-19.
The estimate that nearly 3,000 people would die per day by June was disavowed by the modeler, who said the data was incomplete.
But the report came the same day that another modeling group sharply increased their forecast of how many would die of the disease.
The new estimate of nearly 135,000 deaths by August is more than double what the group was predicting just three weeks ago.
The group, at the University of Washington Institute for Health Metrics and Evaluation, is among those advising the White House on COVID-19 policy.
But it's by no means the only group modeling the pandemic's trajectory. Data analysis blog FiveThirtyEight.com highlights six major groups whose ranges of death toll predictions vary by tens of thousands. Differences among their state-level forecasts are even more significant.
Best guess
In the absence of experience, models provide policymakers with the best guesses available for what the implications of their decisions will be.
But with a virus that scientists discovered less than six months ago, there is a tremendous amount that modelers don't know. And it changes all the time.
"All of these models are dependent entirely upon the assumptions that you put into them," said Mark Roberts, director of the University of Pittsburgh's Public Health Dynamics Lab. "Unfortunately, there's just a tremendous amount of uncertainty about this virus right now and about how it behaves."
For example, scientists have only in the last month or so concluded that people can spread the virus without showing symptoms.
"That was not originally in our model," Roberts said.
Modelers don't agree on how important certain factors are — temperature, for instance.
IHME factors in a 2% decrease in spread for every 1 degree Celsius increase in temperature.
"As heat increases and weather gets warmer, we will see a reduction in circulation," said health metrics sciences professor Ali Mokdad at the University of Washington Institute for Health Metrics and Evaluation. "So, that's a positive."
Roberts is not so sure. It's an option in their model, but "we've turned it off because we don't know how big a difference it makes."
Each model makes assumptions about the effectiveness of policies like social distancing. Much of the data comes from early outbreaks in China. But those results may not hold up elsewhere.
"If the government in China says lock down, people tend to obey," said biologist Katriona Shea at the Penn State University Center for Infectious Disease Dynamics.
In the United States, however, "it's a different political system, and you'll get very different responses to the same management interventions," she said.
New mobility
Some factors are changing constantly, and models have to adjust accordingly.
People are starting to move around more, even in states that have not relaxed restrictions.
But the impact of mobility on the spread of the virus has changed since earlier in the pandemic because people's behavior has changed in other ways.
"Americans, as they are moving about right now, are being very careful and wearing masks and keeping a safe distance," Mokdad said. "The new mobility is different than the old mobility."
For all of these elements, different modelers devise different formulas to calculate the impact and plug different numbers into their equations. They can come to different conclusions.
For policymakers, basing decisions on the wrong model can mean lives lost or resources wasted, Shea said.
So, she is spearheading an effort to "crowdsource" multiple models into tools that not only forecast where the epidemic is headed, but can help officials answer key questions about how to contain it.
While forecasts are important to help prepare for the future, Shea said, "I think even more important is what to do to make the forecast as positive as it possibly can be."
