Does Peer Review Promote Mediocrity?

[Jack Hays]

I think there is no question that peer review will stifle publication of truly unique and revolutionary articles.

That is almost always because those initial articles and findings are totally out of the mainstream and can be seen as borderline crackpot ideas.

BUT- you'll notice that all the cited articles were published- and they were not immediate revolutionary papers- they needed subsequent confirmation.

And I would guess that a control group of rejected papers that were found to be totally on the wrong track outnumbers these seminal papers by 100:1. And that's why peer review works- it keeps most of the nutty and half formed ideas out of the literature.

(Segue to Jack prattling on about Svensmark- and missing the point that the evidence points to the likelihood that it's a futile hypothesis)

That is all well and good, but it's not exactly the point the authors were focused on. Did you see their "significance" note?

"Peer review is an institution of enormous importance for the careers of scientists and the content of published science. The decisions of gatekeepers—editors and peer reviewers—legitimize scientific findings, distribute professional rewards, and influence future research. However, appropriate data to gauge the quality of gatekeeper decision-making in science has rarely been made publicly available. Our research tracks the popularity of rejected and accepted manuscripts at three elite medical journals. We found that editors and reviewers generally made good decisions regarding which manuscripts to promote and reject. However, many highly cited articles were surprisingly rejected. Our research suggests that evaluative strategies that increase the mean quality of published science may also increase the risk of rejecting unconventional or outstanding work."

They are point toward a system that is friendlier to breakthroughs while still keeping "most of the nutty and half formed ideas out of the literature."

 

[Threegoofs]

That is all well and good, but it's not exactly the point the authors were focused on. Did you see their "significance" note?

"Peer review is an institution of enormous importance for the careers of scientists and the content of published science. The decisions of gatekeepers—editors and peer reviewers—legitimize scientific findings, distribute professional rewards, and influence future research. However, appropriate data to gauge the quality of gatekeeper decision-making in science has rarely been made publicly available. Our research tracks the popularity of rejected and accepted manuscripts at three elite medical journals. We found that editors and reviewers generally made good decisions regarding which manuscripts to promote and reject. However, many highly cited articles were surprisingly rejected. Our research suggests that evaluative strategies that increase the mean quality of published science may also increase the risk of rejecting unconventional or outstanding work."

They are point toward a system that is friendlier to breakthroughs while still keeping "most of the nutty and half formed ideas out of the literature."

Oh. NOW you comment.

I'll just point out that it's a conundrum. You can't have literature hate keeping and simultaneously let in the totally original ideas- because only 1-2% of those will ever pan out.

Either way, these things get published.

 

[Jack Hays]

Oh. NOW you comment.

I'll just point out that it's a conundrum. You can't have literature hate keeping and simultaneously let in the totally original ideas- because only 1-2% of those will ever pan out.

Either way, these things get published.

The authors are trying to address the conundrum. The first step is to develop data on the problem. They've done that. Now we'll see what happens.

 

[Threegoofs]

The authors are trying to address the conundrum. The first step is to develop data on the problem. They've done that. Now we'll see what happens.

I can tell you exactly what will happen.

They'll conclude peer review provides an essential gate keeping function, and awareness needs to be placed on the fact that true paradigm shifts usually start as fringe ideas.

But please do not jump to the conclusion that fringe ideas are all paradigm shifts. Very, very few are.

 

[Jack Hays]

I can tell you exactly what will happen.

They'll conclude peer review provides an essential gate keeping function, and awareness needs to be placed on the fact that true paradigm shifts usually start as fringe ideas.

But please do not jump to the conclusion that fringe ideas are all paradigm shifts. Very, very few are.

Of course they are not! Otherwise everyone would recognize them. Merry Christmas to you and yours.:cheers:

 

[Helix]

My position has never changed. You're the one who has been squirming to find a way to assail the unassailable. And yes, those I named are prominent climate scientists who are skeptical of AGW orthodoxy.

well, the scientific community thanks you for your vigilant promotion of and dedication to the honing of the peer review system.

 

[Jack Hays]

well, the scientific community thanks you for your vigilant promotion of and dedication to the honing of the peer review system.

I'll take that as a signal, no matter how ill-tempered, that you recognize you've been on the wrong side of this discussion.

Merry Christmas to you and yours.:2wave:

 

[Helix]

I'll take that as a signal, no matter how ill-tempered, that you recognize you've been on the wrong side of this discussion.

Merry Christmas to you and yours.:2wave:

and once again, you are completely wrong, and you are engaging in poor debating technique. however, i do genuinely hope that you have a good Christmas.

 

[Jack Hays]

and once again, you are completely wrong, and you are engaging in poor debating technique. however, i do genuinely hope that you have a good Christmas.

I didn't expect you to agree.

 

[Threegoofs]

I'll take that as a signal, no matter how ill-tempered, that you recognize you've been on the wrong side of this discussion.

Merry Christmas to you and yours.:2wave:

LOL.

Classic. A true Knight of Python.

Merry Christmas!