Nature has now joined the discussion. Are they attacking peer review too?
[h=2]
Nature admits peer review filters out controversial “champion” papers[/h]
How to separate creative genius from creative mistakes? Not with peer-review. It is a consensus filter.
Classical peer review is a form of scientific gatekeeping (it’s good to see that term recognized in official literature). Unpaid anonymous peer review is useful at filtering out some low quality papers, it is also effective at blocking the controversial ones which later go on to be accepted elsewhere and become cited many times, the paradigm changers.
And the more controversial the topic, presumably, the worse the bias is. What chance would anyone have of getting published if, hypothetically, they found a consequential mathematical error underlying the theory of man-made global warming? Which editors would be brave enough to even send it out for review and risk being called a “denier”? Humans are gregarious social beings, and being in with the herd affects your financial rewards, as well as your social standing. Even high ranking science journal editors are afraid of being called names.
Mark Peplow discusses a new PNAS paper in
Nature:
Using subsequent citations as a proxy for quality, the team found that the journals were good at weeding out dross and publishing solid research. But they failed — quite spectacularly — to pick up the papers that went to on to garner the most citations.
“The shocking thing to me was that the top 14 papers had all been rejected, one of them twice,” says Kyle Siler, a sociologist at the University of Toronto in Canada, who led the study[SUP]
1[/SUP]. The work was published on 22 December in the
Proceedings of the National Academy of Sciences.
There is no formalized sure-fire system to find and reward the creative genius needed for the big leaps in science. Their work must be impeccable logical, but it is an art to cut through human biases to recognise that genius. And art cannot be mandated or controlled. We should never place much confidence in a formalized process, especially one that’s unpaid and anonymous, to spot the papers that will be the most cited 50 years from now.
But the team also found that 772 of the manuscripts were ‘desk rejected’ by at least one of the journals — meaning they were not even sent out for peer review — and that 12 out of the 15 most-cited papers suffered this fate. “This raises the question: are they scared of unconventional research?” says Siler. Given the time and resources involved in peer review, he suggests, top journals that accept just a small percentage of the papers they receive can afford to be risk averse.
For the record:
Siler and his team tapped into a database of manuscripts and reviewer reports held by the University of California, San Francisco, that had been used
in previous studies of the peer-review process.
Anyone who thinks “peer review” is somehow part of the scientific method does not know what science is.
h/t to the brilliant Matthew.
[h=4]REFERENCES[/h] Peplow, Mark (2014) Peer review — reviewed, Top medical journals filter out poor papers but often reject future citation champions. Nature,doi:10.1038/nature.2014.16629 [
Discussion of Siler et al]
Siler, K., Lee, K. & Bero, L. Proc. Natl Acad. Sci. USA
Measuring the effectiveness of scientific gatekeeping (2014).