A black-box version of peer review in which the identities of both authors and reviewers of manuscripts are kept anonymous is receiving significant interest from physicists. Several months into a pilot program for two of its journals, the Institute of Physics (IOP) says that authors of about a fifth of submissions have chosen double-blind peer review, and they are happy with the results. Editors at IOP shared preliminary data last month at the Peer Review Congress in Chicago.
Although IOP is far from the first publisher to toy with double-blind review, its trial comes at a time when scientists are becoming increasingly vocal about improving the peer-review process. Some have advocated for open peer review, in which both authors’ and reviewers’ identities are known; in some such cases, referee reports are published online free for anyone to read. But other authors feel that switching to complete anonymity would ensure a greater degree of objectivity. “There is a culture of openness in most subdisciplines of physics,” says Andrea Taroni, the chief editor of Nature Physics, which has offered double-blind review since 2015. But “author choice is a good thing.”
Traditionally, physics journals and other scholarly publications have adhered to single-blind peer review, in which reviewers’ names are hidden from authors but not vice versa. Knowing authors’ names can help reviewers contextualize papers with the authors’ previous work. But it can also lead to conscious or subconscious bias against unfamiliar researchers, lesser-known institutions, or certain nations and minority groups.
At the start of 2017, in response to feedback from authors, IOP introduced a temporary double-blind option for the journals Materials Research Express and Biomedical Physics & Engineering Express. In the pilot’s first seven months, about 20% of direct submissions to each of the journals were evaluated under the double-blind model. Such submissions were most frequent among authors based in India and relatively common for those from Africa and the Middle East. Some of the researchers from those countries may feel there are prejudices against them in the single-blind model, says Simon Harris, managing editor at IOP Publishing in Bristol, UK.
The IOP data also reveal that the rejection rate for papers submitted under double-blind (70%) is higher than for single-blind (50%). Harris says that may be because there are more low-quality submissions for double-blind review. Other possible explanations include reviewers’ increased objectivity in recognizing inferior submissions or their biased assumption that authors hiding their identities produce lower-quality work. Despite experiencing more rejection, authors participating in a small-scale survey have given the double-blind model high marks, Harris notes, with most suggesting that it is the fairest approach.
The use of completely anonymous peer review in physics goes back decades. The Physical Review journals, which are published by the American Physical Society (APS), offered the double-blind option from 1980 to 2002 but stopped after seeing little uptake. “Historically, double-blind reviews have been difficult to implement,” says APS editor-in-chief Michael Thoennessen. “Referees spend time on trying to guess the authors’ identities, which then might affect their judgment whether they guess correctly or not.”
In 2013 Nature Geoscience and Nature Climate Change started offering double-blind peer review. About 20% of authors making submissions used it, and feedback was positive enough for the process to be rolled out to all Nature-branded journals two years later. Its uptake is only 12% across the 25 journals. Again, the highest rate of use is from authors in India (32%), followed by those in China (22%). The model is particularly tricky in physics, Taroni says, because many researchers post early versions of their papers on the arXiv preprint server.
Since there is no clear consensus on whether open review or single- or double-blind is best, Nature Physics lets referees disclose their identities to authors by signing their reports. In an upcoming trial, journal editors will actively ask reviewers if they would like to share their names with authors. Meanwhile, Harris says IOP will decide whether to adopt double-blind for all the publisher’s 40 or so journals after the pilot ends in January 2018.
