This blog contains the thoughts of the
author rather than representing the work or policy of the Science Media Centre.
There is a quiet revolution taking
place in scientific publishing and almost everyone I speak to thinks it’s long
overdue. One aspect of the revolution is ‘preprints’, the practise of
making scientific papers available to any experts for scrutiny before journal
peer review and publication. Others explain the benefits far better than me but
suffice to say enthusiasts argue that – if adopted widely – preprint could
herald the end of the perverse effects of the dominance of journal impact
factors over scientific careers and speed up the publishing process which can
sometimes see scientists waiting for months or years as they work their way
down the journal hierarchy.
All this sounds great to me and I am
a big fan of bold and disruptive changes which can lead to fundamental culture
change. My reading around work on reproducibility, open access and preprint
make me proud to be part of a scientific community intent on finding ways to
make science better. But I am concerned about how this change might affect the
bit of science publication that we are involved with at the Science Media
Centre. The bit which is all about the way scientific findings find their way
to the wider public and policymakers via the mass media.
This essay is an appeal to the
scientific community – researchers, publishers and communicators – to take
stock and engage in a discussion of the wider impacts of preprint.
Now at this stage I feel the need to
throw in a few caveats and limitations.When I sit down to write I usually know
exactly what I think about a subject which is pretty clearly defined.
Neither is true of preprint. No one yet knows whether this will be widely
embraced and – if so – how things will look at our end of the
chain. Depending on who I speak to we might have nothing to worry about or
lots to worry about. So why write this essay now before our opinions are more
developed and the landscape more certain?
For two reasons. First, because
I think many of those who are driving preprint have not yet considered the
impacts of this exciting development on the wider public and so I hope there is
still all to play for in terms of these discussions. And second, because
more and more people are telling us that preprint is now coming to new subject
areas including biology, medicine and climate change. As many people have
pointed out, preprint has been around in subjects like physics and maths for
many years now and the sky has not fallen in. Very true. But studies in physics
and maths don’t tend to make the front pages of the Sun and Daily Mail, don’t
tend to influence personal behaviour or decisions about new technologies and
are therefore not the core focus of the Science Media Centre. It may be rather
self-regarding but we care about this now because it is about to affect what we
do. Potentially dramatically.
So what exactly is the SMC worried
about and what would we like to discuss with those driving and shaping the
rules of preprint?
The system works (relatively) well
The daily diet of stories for the
average science and health journalist working in national news typically comes
from new findings published each week in the peer-reviewed literature including
journals like the BMJ, Lancet,
Science, Nature, PLoSetc. At present they report these
studies primarily from press releases issued by the journal and the
authors’ universities/research institutes; or from a press conference (at the
SMC or elsewhere) where authors present their findings and highlight
limitations and caveats.
The press releases are issued under
embargo to the time of journal publication, allowing the journalists a few days
to read the paper, call up the authors directly, maybe attend a media
briefing and to seek reactions from independent experts in the field. For
the last 15 years these journalists have also received reactive quotes from
other scientists, via the Science Media Centre. We proactively issue
these comments with the agreement of the journals to help ensure that
time-poor journalists have easy access to the wider context of research in
this field as well as any important caveats and limitations that the wider
public should be aware of.
When you add these routine activities
together and throw in a group of specialist science journalists who generally
strive to produce accurate copy you have the situation we enjoy in the UK
today: a system which includes checks and balances aimed at helping ensure
that what is reported to the wider public is broadly accurate and reliable.
Journalists and scientists both feel this system improves things on their side
of the fence.
Of course it doesn’t always work
perfectly and we still see sensationalism in both science and news coverage.
But as someone absorbed in media reporting of science for 16 years I can tell
you it works better than it used to. There is a good reason why people still
like to hark back two decades to the pill scare or MMR for their go-to examples
of where it all went wrong.
How preprint could disrupt that
Preprint may disrupt this system in
the following way. Instead of new findings being seen first by
journalists when they have been peer-reviewed and are about to be published in
a journal, research may be placed on a publically accessible server at the
same time as it is submitted to a journal – sometimes before. This potentially
means a dramatic increase in the amount of non-peer reviewed findings available
to journalists, should they choose to seek them out; and through them to the
It’s true of course that every busy
science reporter will not be able to scour thousands of papers on preprint
servers looking for a story; and why would they want to when they already have
50 ready-made stories in their inbox.But only one journalist has to
find and report on new findings for it to be ‘old news’ to competing
titles, and even older news when it finally makes it to peer-reviewed
publication. Nature Newsreporters have already proved pretty adept at finding interesting preprints and we’ve already
seen journalists on some nationals reporting on preprints. And what’s to stop media-hungry
authors who are excited about their findings from alerting journalists to their
newly-posted, unverified findings?
Try as I might I cannot find much
about this prospect to calm my nerves. Surely we have to entertain the
possibility of many more new scientific claims being reported in the news media
before they have been subject to any peer review, without the fact-checking
time provided by an embargo, without a measured and cautious press release from
the journal or university, without the benefit of the third party comments
gathered by the SMC. Does any of this worry you too?
The critical point is this: that once
these findings have been reported in one or two national newspapers they cannot
be unreported. If the Times or the BBC report a preprint which claims that
ecigs cause similar harms to cigarettes, or that statins have severe side
effects, then the news editors on the Daily Mail and Telegraph are
unlikely to report a less exciting and toned-down version of the same story
when it’s published. They will either run the same story, or not at all. That
means that the claim that appeared in the press and did the rounds on twitter
may well be the only one that captures public attention. If the research is
ever revised or rejected for publication – as many are – the public will
already have been misinformed. Journalists will not revisit the corrected
version; the damage has been done. We could perhaps live with that when the
subject is gravitational waves. But can we risk that damage to public understanding
on vaccines, climate change, diet or fertility?
Peer review may not be perfect, but
most people agree that one of the good things it tends to do is to tone down
findings and filter out overclaiming. It also sometimes means that papers that
are especially weak and flawed never make it to publication and the public have
been saved a scare story based on flaky evidence. I understand the peer
review process will in many ways be enhanced by preprint and that does excite
me. New research will enjoy the benefit of scrutiny by hundreds of peers, not
just two or three. Terrible papers will hopefully sink under the assault of an
army of scientists pulling them apart while the best papers will rise to the
top. I like all that. But the problem is that the public, who are not qualified
to pull new findings apart, may be hearing about them before any of this
beautiful self-correcting stuff takes place. The stable door will be more
firmly shut on bad research than ever before, but for the public the horse will
have long bolted.
The answer I get when I raise these
kinds of concerns is that this kind of reporting already happens.
Poor quality research already makes it into peer-reviewed publications and then
into the press; non-peer-reviewed abstracts from conferences are already
routinely reported as big news because of unscrupulous over-claiming for
preliminary research. All this is true, but it’s hardly reassuring is it? After
all it’s not often that I lie awake at night thinking that the answer to these
problems is to inject a whole load more non-peer-reviewed research into the
system. The last thing we need is a race to the bottom.
Preprint may also mean less coverage as
well as less quality coverage. Do authors know this?
Another thing that should focus the
minds of scientists and science press officers is the potential impact on the
quantity and impact of the media coverage of new findings. We already know of
cases where unsolicited media coverage of a newsworthy preprint adversely
affected the amount of coverage when it was finally published. As well as the
positive impact of embargos on quality of reporting, the embargo also plays the
equitable function of allowing all the media to run an important story at the
same time. I am conscious that there are critics out there who see the embargo
system as a horribly conformist and controlling system that excludes
journalists working on Sunday papers and discourages original and investigative
reporting. Well maybe. But if we get back to the public interest I would say
that the embargo system wholly benefits the public interest, entitling readers
of The Sun and listeners to Today to
learn of important findings at the same time. Embargoes create a level
I have long reminded scientists and
journal editors not to take to take it for granted that important scientific
findings on everything from climate change to drug safety to mitochondrial
donation are conveyed to a mass audience by responsible science journalists on
a daily basis. The Lancet’s network meta-analysis on
anti-depressants earlier this year made several front pages
including the Sun, Times and Guardian. As a result few people in the UK do not now know
that the weight of good quality evidence on anti-depressants indicates that
they work better than placebo. But what if that study had been found on a
preprint server by an intrepid Guardianreporter 6 weeks before publication? If they had
splashed it on the front page it’s unlikely that other papers would follow up a
scoop by a rival; even less likely they would wait patiently for the
peer-reviewed version. Readers of other newspapers and people who rely on TV
news may well miss the story altogether. I wonder if journal editors and
authors posting on preprint have thought about this. Have they been encouraged
to consider that by putting their findings in the public domain they may also
inadvertently be limiting the amount and impact of media coverage when their
findings are published? Strong new evidence in the public interest deserves
wide, loud coverage; I am worried that a preprint model may jeopardise that.
Embargoes fulfil another important
purpose of course: they provide journalists with breathing space to weigh up,
summarise and seek critique to new findings. In a world of 24 hour rolling news
where journalists are rewarded for breaking stories even minutes before the
competition, the pressure to rush to publish is stronger than ever. The embargo
system keeps that in check. Journalists generally find out about new research
several days before they may publish. They can break those embargoes of course;
but they don’t. The real reason for that is because they like them. Far from
curtailing their investigative instincts, they recognise that embargoes give
them the opportunity to seek the thoughts of experts in the field. Every week
at the SMC we see attractive stories tempered by 3rd party comment and
sometimes sinking without trace – precisely because they have been critiqued by
the scientific community. Journalists don’t want to write stories built on bad
science, and the public shouldn’t be fed dodgy claims presented as fact.
Embargoes allow this system to flourish. Without them stories will be hurriedly
written, and scientists and journalists will have no chance to tackle weak or
false claims before they are presented to an unsuspecting public.
Some ideas for discussion
Embargoes may yet have a place in a
preprint world – but not if we throw them out prematurely
The received wisdom on embargoes in
our discussions on preprint is that no science press officer will be able to
apply an embargo to a published paper when it is already in the public domain
as a preprint. Some have even suggested that it is morally untenable or
unethical to impose an embargo restricting journalists from running a story
which is ostensibly already out there. This is being repeated as though it’s a self-evident
truth – but I want to challenge it.
I can clearly see the arguments as to
why embargos will be difficult to enforce with preprint but there are strong
moral and practical arguments for ensuring that new findings with a significant
public impact are managed in a way that maximises good quality reporting.
Before abandoning the entire embargo system with a somewhat fatalistic shrug –
‘what else can we do?’ – I suggest we spend some time thinking about our
Already I can see some discussion
points here. Could we have different criteria for publications and save
the embargo system for papers with clinical or public health implications? Or
could we argue to journalists that embargoing a final paper that has now
completed peer review and been accepted by a journal is still legitimate
because it will differ in meaningful ways to the paper that is sitting on
preprint? We should remember here that journalists enjoy benefits from embargos
too. If the BMJ or JAMA treat
their published papers as qualitatively distinct from preprints then the
journal, the authors and the media may benefit from us not giving up on
the embargo quite so casually. At least it’s worth discussing?
Best Practice guidelines
As I said, one of the arguments made
by preprint fans is that most of what I worry about already happens. Science
gets into the news before it’s published all the time for a variety of reasons,
and poor science makes it into journals far too often. Moreover, I am telling
you that we routinely minimise the damage from these through the combined
efforts of responsible science press officers, good science journalists and the
SMC’s ‘Roundups’ service. Surely we just need to adapt our current approaches
to the new kid on the block?
Maybe. But I still think we need to
use this time to thrash out best practice and agree what the new rules should
look like. One discussion point would be to agree that good practice would
mean universities not issuing press
releases at preprint stage. That would
acknowledge that the aim of preprint is to open up the process to allow experts
to scrutinise and learn from their peers’ work – and not to seek publicity or
promote it to the wider public as accepted findings. We could play with
terminology here and start using terms that clearly differentiate preprints
from accepted papers. We could also discuss whether press officers should
routinely issue guidelines to their scientists or authors, urging them not to
promote preprint findings to the media before they are accepted for publication
(some journals are already taking steps in this direction).
Are there other ways?
Or is there something we have not
thought of that could get us round these new realities with minimum adverse
effects? Maybe journals can no longer embargo new studies but the
press officers would still know when they were planning to issue an ‘immediate
release’ and could prepare additional materials to help journalists get round
the lack of time. We have already asked journal press officers whether they
might also be able to give us advance notice so that journalists could continue
to see third party comments from leading scientists at the same time as they
see the news release. Some certainly seem open to considering that. And it’s
possible, of course, that journalists would be content to persist with a system
of embargoed press releases – even in the full knowledge that an early,
preprint version of the work may exist on a public website somewhere. After
all, an embargo will hold as long as everyone agrees to hold it. At the
very least we must seek their views.
What has really struck me in my discussions about preprint is that the changes being made to a part of the system that was not working are set to have profound knock on effects on another part of the system that works and serves science well. The challenge here is to fix one end without losing the gains we have made in reporting findings to the public in an accurate and measured way. Science is rightly proud of its reputation for informing the public based on strong evidence. Let’s not destroy that in pursuit of preprint – however great the prize.