Last October, I wrote that COVID-19 had created a stampede of shoddy research. Little has changed in the interim. Putting all hands to the pump might feel appropriate in a crisis, but during a global public health emergency, rushing headlong into the scholarly frontline is anything but okay. Frankly, it is dangerous.
Scholarship, by its nature, is difficult. Fast science is never good science. Even the development of COVID-19 vaccines — often described as setting records for research speed — took much more time than you think.
The process of producing the vaccines involved hundreds of scientists conducting thousands of hours of painstaking lab work, and then testing tens of thousands of volunteer participants over several months. The overall delivery time might appear impressive, but the total investment of person-hours was huge. And the endeavour benefited from years of prior groundwork carefully completed in anticipation of exactly this type of crisis.
The scale of this science was unprecedented in human history. But in no way was it ‘rushed’.
I’m not so sure the same can be said about all the other scholarly contributions we have witnessed during this pandemic, especially where the social and psychological sciences have been concerned.
* * *
According to one estimate, around 200,000 journal articles on COVID-19 were published in 2020. That’s roughly 550 articles per day. Or, if you want to talk about speed, it’s one paper every two-and-a-half minutes. Day and night. 24/7.
Investigators, reviewers, editors, and readers are processing more and more information with ever lessening bandwidth. Our systems of peer review were never designed for this level of activity. In fact, they were never ‘designed’ at all. Peer-reviewing evolved organically as an amateur honour system intended to bolster the credibility of academic journals. Its limitations were apparent long before COVID struck.
Perhaps unsurprisingly, given the pressure on the system, many academics are now opting to post their work to preprint servers, where they are available to the public, and the media, without any peer review. In 2020, the first full year of the pandemic, researchers posted a whopping 30,000 such preprints on the subject of COVID-19 alone. More than two-thirds of all preprints posted to the medRxiv server were on topics related to the coronavirus.
And then there is the problem of ‘science by press release.’ With attention so focused on the pandemic, there has been an upsurge of news coverage featuring scientific assertions that are little more than supposition repackaged as fact. Many of these relate to findings from research that has never been published at all.
* * *
Vaccination psychology has presented a particularly lucrative bandwagon. Social scientists have much to say about what people think about vaccines, and about how vaccine hesitancy might be addressed. There is an extensive established literature on the topic. For example, the psychological research on how people gauge risk is especially relevant.
However, much media commentary on COVID vaccines has been speculative and opinion-based, or derived from vaguely described, unpublished, or unreviewed research studies.
For example, this week there was extensive media coverage of a vaccine hesitancy survey conducted in the United States, prompted by a university press release. The research described has not been peer-reviewed. Instead, it was submitted to medRxiv as a preprint. What this means is that the study, and its findings, have not gone through the standard quality-control filtering process that regular research is supposed to undergo.
The medRxiv website states clearly that the study has “not been peer-reviewed,” is “yet to be evaluated,” and “should not be used” to guide practice. However, the relevant university press release includes none of these caveats. Instead, it describes the findings as being announced “ahead of peer-reviewed publication,” which seems just a little presumptuous — by December 2020, only a quarter of medRxiv’s COVID preprints actually ended up published in a journal.
The study might be great and it certainly has a large sample (with more than 700,000 respondents being surveyed at three separate time-points). But we do not know that the survey is sound because the work has not been independently evaluated.
The practice of issuing a press statement anyway is the very definition of ‘science by press release.’
* * *
The second bandwagon has been Long COVID. This complex condition is still poorly understood, but that has not prevented some psychosocial practitioners from asserting that they certainly know what is going on.
For example, one group of academics wrote to the Guardian to say that Long COVID can be treated using graded exercise therapy because “we know that graded exercise therapy is an effective treatment for chronic fatigue syndrome (or ME)”. The problem, of course, is that they cannot realistically “know” such a thing. No substantive scientific evidence supports such a claim. In fact, when the UK’s National Institute for Health and Care Excellence recently reviewed the relevant research, they concluded that the actual evidence showed quite the opposite. Based on the science, NICE warned, GET should never be used for such a purpose.
Another academic, a doctor no less, told the Guardian (along with just about every other media outlet in Britain) that he had cured himself of Long COVID using the power of positive thoughts. The problem here was that the particular form of Long COVID he experienced seemed rather, well, unique. By his own admission, he was able to complete 5k walks, take bike rides, and even go scuba-diving in Grenada, all whilst apparently still unwell.
Of course, I am glad this man recovered and I wish him good luck. But it is somewhat scandalous that the media should facilitate him in parlaying his idiosyncratic experience into what has become little more than a modern-day travelling medicine show. His much peddled ‘mind-over-matter’ mantra serves only to stigmatise those thousands of Long COVID sufferers whose illnesses would surely benefit from the input of actual science-based medicine.
* * *
Academic peer review is far from perfect. In fact, it is frequently terrible. Nonetheless, a complete absence of peer review is hardly the solution.
Claims in the media about cures for Long COVID, reasons for vaccine hesitancy, the consequences of lockdowns, or public attitudes toward restrictions can have very serious consequences. They shape people’s thinking and decision-making about the pandemic, as well as their sentiments toward science itself.
Unreliable factoids, unfiltered opinions, and unreviewed research should never be reported as news. Journalists who puportedly specialise in reporting on science and health really should know better.
But it is the scientists themselves who should know best of all. Everything starts with them. ‘Science by press release’ is clearly dubious, and the offering of opinions as facts is just unethical. There is no excuse for cutting corners, even if the whole world — or the university press office — is gagging for COVID-related news items.
A global pandemic should challenge us to apply rigorous thinking to help solve the world’s problems. It should not be seen as a sexy topic that can be commandeered into your next colour-by-numbers survey study, an opportunity to advertise your therapeutic indispensibility, or a cheap way to add topical lustre to your academic CV.
It’s truly time we flattened the curve of shoddy COVID science.
The post Time to flatten the curve of shoddy COVID scholarship appeared first on The Science Bit.
