The scholarly evaluation process, commonly referred to as peer review, represents a foundational element of contemporary academic pursuits. Prior to the dissemination of new scholarly work within academic periodicals, subject matter experts meticulously examine the empirical data, research methodologies, and argumentative structures to ascertain their validity and coherence.
Nevertheless, a considerable number of scholars, evaluators, and editorial staff encounter significant difficulties with the prevailing mechanisms of the modern peer review system. This system is frequently characterized by protracted timelines, a lack of transparency, and preferential treatment, all while relying on the unpaid contributions of academics who are already overburdened with responsibilities.
In the preceding month, one of the co-authors (Kelly-Ann Allen) articulated her profound dissatisfaction with the challenges encountered in securing peer reviewers via the Twitter platform. Following an extensive outpouring of hundreds of responses, a comprehensive compilation of criticisms regarding peer review, alongside proposals for its enhancement, was amassed through collective input.
As a journal editor, reading that people are publishing and not participating in reviewing is hard to hear. It is becoming increasingly difficult to find reviewers and this is creating significant delays in publishing timelines. #AcademicChatter @AcademicChatter https://t.co/FMFeqFnsrm
— Kelly-Ann Allen, PhD FAPS (@drkellyallen) June 6, 2022
The recommendations put forth for scholarly journals, publishing houses, and academic institutions indicate ample opportunities exist to foster a more accountable, equitable, and inclusive peer review paradigm. Our comprehensive findings have been concisely summarized herein.
Three Principal Obstacles Confronting Peer Review
The peer review infrastructure is presently grappling with three primary challenges.
Primarily, the peer review process can be perceived as exploitative.
A substantial proportion of entities that publish academic journals derive financial gains from subscription fees and product sales. Conversely, the authors, editors, and peer reviewers typically dedicate their time and expertise on a voluntary basis, effectively rendering their contributions as pro bono work.
While peer review is frequently conceptualized as a collaborative endeavor undertaken by the academic community, in practical terms, a disproportionately small segment of researchers shoulders the bulk of this workload. An investigation into biomedical journals revealed that, in 2015, a mere 20 percent of researchers were responsible for completing approximately 94 percent of the peer reviews.
Peer Review Operates as a ‘Black Box’
The secondary challenge pertains to the pervasive lack of transparency inherent in the peer review methodology.
Peer review is predominantly conducted under conditions of anonymity; researchers are unaware of the identities of their evaluators, and conversely, reviewers do not know whose work they are assessing. Although this anonymity can facilitate candid feedback, it simultaneously renders the process less open to public scrutiny and accountability.
This inherent obscurity may also stifle open dialogue, perpetuate inherent biases, and consequently diminish the overall caliber of the evaluations provided.
Peer Review Suffers from Protracted Timelines
The tertiary challenge concerns the protracted temporal nature of the peer review process.
Upon a researcher’s submission of a manuscript to a journal, should it successfully navigate the initial screening stages of rejection, the author may then endure a considerable waiting period for review and subsequent publication. It is not an infrequent occurrence for research findings to be published a year or more after their initial submission.
Academics suggest that these delays are typically attributable to an insufficiency of available reviewers. Numerous academics report that demanding professional obligations act as a deterrent to their participation in peer review, a situation that has been exacerbated since the advent of the COVID-19 pandemic.
Furthermore, it has been observed that a significant number of journals depend heavily on reviewers based in the United States and Europe, thereby constricting the breadth and diversity of the reviewer pool.
Can the Peer Review System Be Rectified?
Consequently, what remedial actions can be implemented? The majority of the constructive proposals emerging from the extensive Twitter discourse previously mentioned could be categorized into three principal areas.
Firstly, a prevalent suggestion was the implementation of enhanced incentives for the undertaking of peer reviews.
Such incentives might encompass financial remuneration for reviewers by publishers (as is already practiced by the journals of the American Economic Association) or the allocation of a portion of profits to research departments. Journals could also extend benefits such as complimentary subscriptions, vouchers for publication fee waivers, or expedited review processes for their evaluators.
However, it is important to acknowledge that the introduction of incentives by journals could potentially give rise to new complications.
An alternative recommendation involves academic institutions playing a more active role in recognizing peer review as an integral component of the academic workload, and perhaps offering commendations to individuals who make exceptional contributions to this process.
Some contributors to the Twitter discussion advocated for tenured faculty to review a predetermined number of articles annually. Others posited that greater support should be directed towards non-profit scholarly journals, particularly in light of a recent study that indicated approximately 140 journals in Australia alone ceased publication between 2011 and 2021.
The consensus among most respondents was that the avoidance of conflicts of interest is paramount. Certain individuals proposed the establishment of expert databases to facilitate the identification of suitably qualified reviewers.
Employing More Inclusive Strategies for Peer Reviewer Recruitment
A significant number of respondents also proposed that journals could refine their reviewer recruitment methodologies and the scope of the tasks assigned. Expert reviewers could be identified based on their proficiency in specific methodologies or content areas, and their review efforts could be focused on those particular aspects rather than encompassing the entirety of the manuscript.
Respondents further argued that journals should enhance their efforts to tailor invitations, targeting the most pertinent specialists and streamlining the process for accepting or declining such requests.
Others expressed the view that a broader demographic of reviewers should be engaged, including early-career academics, doctoral candidates, professionals in related industries, and retired experts. Providing enhanced peer review training for graduate students, coupled with increased representation for women and underrepresented minority groups, would represent a positive initial step.
Reconsidering Double-Blind Peer Review
Certain respondents highlighted an emerging trend towards more transparent peer review processes, which could foster a more humanistic and open approach to scholarly evaluation. For instance, Royal Society Open Science makes public all editorial decisions, reviewer correspondence, and voluntary disclosures of reviewer identities.
Another suggestion aimed at accelerating the publication trajectory involved assigning higher priority to research that addresses time-sensitive issues.
Potential Avenues for Improvement
The overarching sentiment derived from the prodigious response to a singular tweet underscores the imperative for systemic reforms within the peer review framework.
There is an abundant supply of innovative concepts for refining the process to the benefit of scholars and the broader public. Nevertheless, the onus rests upon journals, publishers, and academic institutions to translate these concepts into practical implementation, thereby cultivating a more accountable, equitable, and inclusive system.
The authors extend their gratitude to Emily Rainsford, David V. Smith, and Yumin Lu for their contributions to the original publication titled “Towards improving peer review: Crowd-sourced insights from Twitter.”
