After weeks of waiting, you finally receive the email from Nature Food Physics with the reviews of your submission “A theory of the nonuniform browning of toast.” Hope turns to despair as you read the editor’s cover letter saying that because the reviews contain sufficiently strong criticism, your paper cannot be accepted. Then you read the reviews and discover that Referee A said, “This is a wonderful paper, full of interesting new results that will surely be of interest to a wide audience. I am particularly impressed with equation 7 and its consequences and expect it to have broad applicability in physics.”
So what’s the problem? Well, Referee B said, “I fail to see any great significance in the results presented, and I doubt the paper will be of broad interest. In addition, the result, equation 7, is wrong, calling into question the entirety of the subsequent results.” Besides making you wonder why the editor listened to Referee B and not A, or why he did not try to figure out whether equation 7 really is wrong, this hypothetical scenario points to a deeper problem.
I’ve served as an editor for both Physical Review Letters and Reviews of Modern Physics. At those publications and at most other journals I regularly publish in, including Proceedings of the National Academy of Sciences, Journal of the Royal Society Interface, and Journal of Fluid Mechanics, the peer-review system suffers from a communication problem: With few exceptions, there is no mechanism for the referees and the editor to discuss the paper and arrive at a consensus recommendation before reviews are sent to the authors. Instead, the initial recommendation is based on the editor’s implicit averaging of the reports. The author is left to counterargue what might have been a referee’s error or try to reconcile often contradictory reports. In subsequent rounds of review, referees often see earlier reports, but still they do not communicate with each other to build consensus regarding the current review. We who spend our professional lives dealing with this deeply flawed process from either side deserve better.
Remarkably, the problem has been solved by a relatively new, high-profile, open-access journal in the life sciences: eLife. Launched in 2012, the journal is a joint effort of the Max Planck Society in Germany, the Wellcome Trust in the UK, and the Howard Hughes Medical Institute in the US. It is meant as a direct challenge to the troika of journals— Science, Nature, and Cell—that dominate the life sciences; the first two also have a strong presence in certain areas of physics. The editorial decision-making process at eLife is vastly different from the process at those three journals, which mostly rely on in-house editors rather than practicing scientists to initially cull submissions before sending only a minority out to review. But eLife’s reviewing process in particular is one that publishers of physics journals might do well to adopt.
I first learned about the journal from colleagues connected with the Wellcome Trust, from which I receive long-term funding. I’ve published several articles in eLife1–3 and acted as a referee for it, but I have no formal association with the journal. I have spoken with many people about the eLife review process and can report that even those whose papers were ultimately rejected spoke highly of it.
The essence of the eLife review process is an online discussion between the referees and the handling editor of a paper so that they arrive at a single consensus report—a “decision letter”—that is sent to the authors. The discussion comprises a series of posts behind the journal’s firewall, where the referees and editor are known to each other. The individual reviews are generally not sent to the authors, although comments they contain may be incorporated into the decision letter where appropriate.
Once the referees and editor have reached a consensus, the decision letter lays out the assessment—whether the paper requires minor changes, needs a major revision, or is rejected. For papers requiring changes, the letter describes what needs to be done to make it acceptable for publication. If the paper is accepted, the letter and the authors’ reply are published along with the paper. If the paper is rejected, the process is like that of most journals: The editor typically includes with the decision letter the full, original referee reports so the authors can see all the concerns raised.
The advantages of this system are obvious. If the referees differ on a technical point, they discuss it and arrive at a single point of view—for example, clarification is needed or a problem needs addressing. Publication standards for eLife are very high with regard to importance, broad interest, and novelty, and there, too, the referees and editor resolve any disagreement and then speak with one voice to the authors. The fact that the reviewers are known to each other during the online review process naturally tends to enforce both higher standards and greater civility than would be the case with anonymity. Because there is a unified editorial response, the authors spend less time doing additional experiments or responding to contradictory referee reports.
The disadvantages, I think, are few. The online discussion can certainly take more time. It requires more up-front work from the editors, who must coordinate the process. Ultimately, though, less work is involved in subsequent reviewing rounds because the editor can usually decide on the suitability of the revised paper without sending it back to the referees. More work is certainly required from referees, who must engage in the online discussion, but they will likely do a better job precisely because of that. Overall, the process is much more satisfying to all involved.
I leave it as a challenge to the physics community to adopt this review process. I have enough experience with the editorial issues confronted by journals to know that such a change would involve considerable work. So let us start with a single journal—say, Physical Review Letters—and see if we can make the review process work better.
For discussions on this subject and feedback on drafts of this article, I am grateful to Randy Schekman and Andy Collings, respectively editor-in-chief and executive editor of eLife, and to my colleague Eric Lauga. This work was supported in part by the Wellcome Trust and the UK’s Engineering and Physical Sciences Research Council.
