Why scientists change their mind and disagree

If you’ve been the daily news cycle during the coronavirus pandemic, you probably noticed circumstances where scientists seemed reluctant to share information, debated the latest research on social media or downright changed their views. 

In our culture, we often hold politicians, corporate executives and other leaders accountable for the consistency of their positions. In political debates, candidates will often point out on the debate stage that a rival swung to the left or right over a controversial issue. It suggests a lack of authenticity, or even careerism, and indicates that they can’t be trusted to do what’s right for their constituents.

In the scientific world, it’s expected that even the highest-ranking academics will evolve their thinking — and many have done so during this Covid-19 pandemic.

But some scientists fear that the public doesn’t understand this, and is losing faith in scientists who change their minds. And that’s having real consequences on the front lines.

Dr. Megan Ranney, an emergency physician who works at the Rhode Island Hospital, said some patients are coming into her emergency department refusing to wear masks. When she prompted them to wear one, they often told her that public health authorities like the World Health Organization and the CDC initially advised against wearing masks, saying there was little evidence that it would help prevent people from getting sick.

That recommendation later changed, as studies began to show evidence that people with no symptoms might be spreading the disease. Now, both organizations encourage all people in public to wear masks, including cloth-based coverings, to prevent the disease from spreading — exactly what citizens in some countries, like Hong Kong and Japan, had guessed during the early days of the pandemic based on past experiences. 

But as Ranney pointed out in an interview with CNBC, it’s “part of the process” that leading public health authorities would adapt their thinking based on new information. 

Carl Bergstrom, a biology professor of the University of Washington and an author of a book about misinformation, explained that very little was known about the virus back in January and February. So infectious disease specialists and epidemiologists had to do their best without much data at their fingertips.

Even today, notes Dr. Bergstrom, there isn’t always a clear answer on important metrics like the case fatality rate (Dr. Bergstrom provided a range, when asked about that, and not an exact percentage). Sometimes the only response is “it depends,” or the even less satisfying “we’re still figuring that out.” That can be difficult to hear when the public is searching for answers, and policymakers are looking for clear advice to pass on to their constituents.

“When you take a completely novel virus, you are starting out from a position of by default knowing nothing,” Dr. Bergstrom explained. “You can at best make guesses based on what you know about previous coronaviruses and prior outbreaks of other respiratory viruses.”

As a pandemic progresses, scientists will get more data as more cases occur. “That gives us more time to do basic investigation into the molecular biology of the virus and the interaction between the virus and host,” he explained. “You get more opportunity to watch how transmission works. And you come up with new conclusions based on more evidence, and then you make those public because it’s the best of what you know.”

Others in the community say that it’s even a badge of honor for a scientist to update their thinking when confronted with new evidence. Vinay Prasad, a hematologist-oncologist and Associate Professor of Medicine at the University of California San Francisco, said that best scientists are “continually re-evaluating themselves to see what we got right and what we got wrong.”

As he put it: “It’s a high mark to be able to say, ‘I’m going to change my mind’.” 

With the situation moving so quickly, scientists are rushing to publish papers before they’re peer-reviewed. Those papers are increasingly getting picked apart on social media by communities of their peers — a process that previously would have happened behind closed doors. 

The so-called “preprint servers” like bioRxiv and medRxiv feature research that is disseminated far more rapidly than the usual peer-review process, which can take weeks or months. Consumers are now witness to these discussions and occasional fiery disagreements. 

One particularly noteworthy debate during this pandemic concerned a group of academics, including Dr. John Ioannides, at Stanford University, who have consistently argued there’s a lack of evidence to support shelter-in-place orders. That same group published research via one of these preprint servers indicating that the virus might be more prevalent than initially believed, and therefore potentially less deadly. 

It was pilloried by other scientists on Twitter and other social media platforms and picked apart for the problematic methodologies.  

“These discussions used to take place over email or by phone,” Bergstrom said. “Sometimes there are just different groups working on the same problem, but with a different hypothesis or a different theoretical framework,” he added. “So if you see scientists arguing, it doesn’t necessarily mean that anyone is a bad actor.”

Prasad believes that scientists are right to point out flaws in data or methodology, particularly if the paper has been published in a preprint server or the conclusions that the public are jumping to might be dangerous. But he takes exception with the personal attacks he’s seen on social media. 

For consumers without scientific training, he notes, it can be extremely challenging to determine the true experts in a field, especially when a person’s credentials appear to be solid. He suggests looking a researcher’s publication history, but recognizes that not everyone has time to do that.

If you spot scientists disagreeing, he notes, recognize that it’s somewhat normal — especially given that the stakes are so high right now.

“The more eyeballs on the paper, the more likely it is to get critiqued,” he said. “Because of Covid-19, someone has turned up speed on factory line and it’s not pretty sometimes.”

“Science is under pressure and you’re seeing how the sausage is made,” agreed Bergstrom. 

When the worst of the crisis is over, there will certainly be an opportunity to look back and reflect on some of the mistakes that were made along the way. 

Dr. Prasad said that the scientific community will likely engage in a process of its own to do just that. 

“When the dust settles and we’re a couple of years out, I think it will be a useful exercise to evaluate what we got right. Were they the best policy decisions for the evidence that was available at the time? I don’t think you get off the hook for views that are totally wrong,” he said. 

Timothy Caulfield, the Canadian professor of law at the University of Alberta, differentiates between cases where a scientist changes their mind based on new data, and circumstances where someone misrepresented their work, or falsified data. 

If it’s an inadvertent error, he explained, the research should be retracted with an explanation of the issue, and that should be recirculated to the public. “With so much pressure to move quickly, mistakes seem likely to happen — particularly in preprints. So the scientific community and the media need to take great care in how all of this is reported,” he said. 

Caulfield notes that policy-decisions are changing, but that doesn’t mean that public health shouldn’t be trusted.

He describes the the policy around masks specifically as a “profound communication challenge.”

“Public health sometimes need to adopt positions, even if the evidence isn’t robust,” he said. “And from a policy perspective, those positions need to be championed.”

“But that doesn’t mean that the scientific community should stop talking about the evidence,” he added. “You don’t want to discourage open, honest debate.”