When I submit a paper to a scientific journal, it is reviewed by experts in the field before it is published. The experts might say that the paper should be published as it is. They might say that the paper should be published, but certain changes should be made to either make it more consistent with what we know or to provide better context for understanding the results that are being presented. The reviewers might say that the article shouldn’t be published, because it contains poor science or because the results aren’t important. This is the process of peer review.
I have been on both sides. I have submitted papers for publication that went through peer review, and I have been a peer reviewer for scientific journals in my area of expertise. Although neither process is particularly enjoyable, I think it is a very important part of science. Indeed, even though it is not required, I have all of my science texts reviewed by at least two experts before they are sent to the publisher. These experts often catch errors that I have made, and they also add valuable comments that allow me to improve the way I discuss certain topics. My textbooks are significantly better because they are peer-reviewed.
Of course, there is also a problem with peer review. It can lead to the suppression of scientific results for unscientific reasons. I have experienced this personally. I recall one paper that my collaborators and I submitted to a nuclear physics journal. It was rejected by the peer reviewers, and the journal said that it would not be published. We looked at the reports of the peer reviewers and showed the editor that the reviewers’ objections were without merit. The journal agreed to send it to two more peer reviewers, who both accepted it with only minor corrections.
We later learned that the two peer reviewers who rejected the paper were from research teams who had published papers that our results contradicted. They rejected the paper not because it was poor, but because they didn’t want their results challenged. Fortunately, it was easy for us to show that their rejections didn’t make sense, because their review reports clearly demonstrated the weakness of their case. They apparently thought that since these reports are confidential (shared only with the editor and the team trying to publish the paper), they could use technical jargon to convince the editor that they were right, and no one would be the wiser.
That’s the dark side of peer review. Fortunately, there is a new trend in scientific publication, and if it catches on, it will greatly diminish this dark side. It’s called open peer review.
The Royal Society (the oldest independent scientific academy in the world) has been doing this since 2014. If the authors and reviewers agree, the journal Royal Society Open Science will publish a scientific paper along with the relevant parts of the peer reviewers’ comments. That way, all readers will see not only what the authors think, but also what the peer reviewers think. This seems to be catching on, because the Royal Society says that more than half of the papers published in Royal Society Open Science have open peer review. The numbers vary with discipline, however. About 85% of their physics papers were published with open peer review, while slightly less than 50% of their genetics papers were published that way. Other journals like the British Medical Journal and PloS ONE have been experimenting with open peer review as well.
I do hope this becomes more and more popular. Peer review is a very important part of science, but as we know, it is becoming more and more popular to weaponize science for political and social causes. Confidential peer review makes this weaponization of science easier, while open peer review makes it harder. In addition, I like the idea of reading a scientific paper and then reading what some experts in the field think of it.
Now before I finish, I want to make it clear that peer review is not a perfect system, and open peer review will only help it to a certain extent. There are still times where even open peer review will lead to the suppression of good science. When a paper challenges the “scientific consensus,” for example, it might still be blocked, regardless of its merits. Nevertheless, open peer review seems better than the system that currently exists, and I think that, on balance, it is a positive step for science.
I have no expertise whatsoever in science, but I have experience in education, and in my experience the less the experts control the flow of information the better. The concern for “solid” or accurate materiel so often stunts growth as opinions, practices, and probabilities become dogma. I think it will certainly benefit science to allow the winnowing process of peer-review to be made public. That way, the public has the opportunity to discern for themselves if the review was legitimate.
From a Harvard Business Review article:
“Bringing in ideas from analogous fields turns out to be a potential source of radical innovation. When you’re working on a problem and you pool insights from analogous areas, you’re likely to get significantly greater novelty in the proposed solutions, for two reasons: People versed in analogous fields can draw on different pools of knowledge, and they’re not mentally constrained by existing, “known” solutions to the problem in the target field. The greater the distance between the problem and the analogous field, the greater the novelty of the solutions.”
not sure I understand all the words but i think it means people from other fields can actually be helpful. The closed peer-review system eliminates that possibility – or should I say opportunity. I know there is much more to the issues but from my perspective there is a foundational idea under-girding it that seems to be based on intellectual, well, snobbery. Not that individual experts are, rather it’s systemic. I am glad to hear about the open peer review, I agree it is a good step.