When some subject attracts controversy, there is more to it than mere disagreement. Disagreement need not lead to controversy if the disagreeing parties understand and have learned to live with each other’s point of view. Controversy arises when there is some unresolved tension to be worked out.
The subject of ‘open science’ still attracts controversy because there is no settled coexistence of ‘open’ and ‘closed’ models of science. There is disagreement over just what the “open” in open science should be taken to mean and over what type or degree of openness is the best for science. Those who are enthusiastic about greater openness tend to focus on themes of transparency, accountability, fairness in getting research published and, of course, “free” access to data. Those who still feel skeptical about open science tend to focus on the need to maintain standards of quality and reliability. Because the open science debate largely remains one that is conducted by science professionals for science professionals, tension arises over the extent to which the opening up of science should be allowed to disrupt the established norms of professionalised scientific practise.
One area where the effects of this tension can be see is in attitudes to the opening of peer review of research reports. A recent high-profile retraction of scientific papers that apparently drove one of the researchers involved to suicide, led to calls to open up the processes of peer review[*], but the editor of the journal concerned said that, while this had been considered, “the disadvantages — which include potential misinterpretations and the desire of many referees to keep their comments confidential — have prevented the journal from embracing this”[*]. Clearly, there are conflicting motivations here. Regardless of the effects on overall research quality, a major barrier to opening up peer review is the perceived desire of referees to preserve the established norm of anonymity.
In practise, peer review is a process of negotiation between the authors of a proposed research report, editors of the journal to which it has been submitted and reviewers selected on the basis that they are well-informed representatives of the eventual audience for the report. Authors want to get their report published in a journal with a ‘brand’ reputation that attracts the right sort of reader (people who’ll cite the paper, basically). Editors want papers that will reinforce the journal’s reputation for bringing out quality publications of interest to its readership.
Peer review is widely identified as a cornerstone of quality assurance in institutional science, most people readily admit that it has very obvious faults. Review is entrusted to a small number of individuals whose competence and trustworthiness are judged only subjectively by the editors. While reviewers are supposedly chosen on the basis that they possess a strong understanding of what quality means in relation to the relevant field of research and have a commitment to seeing it maintained, they may have other motives as well, such as getting to see new research results before everyone else or even seeking to influence what results others get to see. Another effect of institutional peer review is that acceptance of a paper for publication itself signals to readers that the work described is worthy of their attention and that the conclusions drawn by the authors are respectable. Individual readers are free to take contrary views, of course, but by doing so, they risk marking themselves as outsiders or even cranks if it’s not evident that many others feel the same way. Even when a post-publication debate takes place on the significance of a paper, there is not usually any mechanism for making the content of the debate a necessary part of reading the paper itself. The interpretations negotiated during the peer review process and set out in the published paper remain the ‘official’ position unless it turns out that the paper contains errors or misdemeanours serious enough to warrant retraction of the paper.
No doubt, there are circumstances where complete retraction is appropriate, but in many cases a discussion of what seems wrong and what remains good about the research report might be quite possible. There are plenty of reasons to believe that far more papers are in need of this kind of evaluation than are ever retracted [*]). There is at least one online forum (PubPeer) that tries to provide this kind of facility. But, it is notable that the people who make PubPeer say they have collectively decided to remain anonymous in order to avoid “circumstances in which involvement with the site might produce negative effects on their scientific careers”[*]. Clearly, there is real tension over the idea of open peer review where just anyone can criticise a research report and be identified for doing so.
Perhaps this tension will only resolve itself when an ‘open’ model of science abandons the idea of authoritative research statements as represented by the ‘scientific paper’ altogether and instead sees results only as stimulus to imagination that engenders debate and motivates further research action.