Peer review in scientific publishing: Current practice, guidelines, relevancy, and way forward

INTRODUCTION

Peer review is considered the cornerstone of scientific publishing because it provides a way to assess the quality and validity of scientific research before it is published.^[1] In peer review, a panel of experts in the same field as the author evaluates the research, its methods, and its conclusions, offering suggestions for improvement and checking for errors, biases, or inaccuracies. This process helps to ensure that only the well-conducted and trustworthy studies are published and provide a measure of quality control for scientific literature.^[2] This ultimately protects the integrity of the scientific enterprise and enhances the credibility and reliability of scientific research.^[3] However, the early carrier researchers or academicians do not get enough exposure, information, or motivation for conducting a peer review. They may have lots of questions in their mind [Figure 1]. Hence, in this narrative review, we discuss about the current practice of peer review, its types, advantages and limitations, methods and online resources to learn reviewing, necessity at the level of reviewers, and the way forward.

Close

Figure 1:

A carton showing queries of researchers about peer review. The cartoon image was obtained from Pixbay.com with a creative commons license and was edited for this figure.

Export to PPT

REVIEW PROCESS

The steps of review of a research article start after submission of an article to the journal either through online manuscript submission system or by other electronic medium. Taking printed manuscript is rarely practiced nowadays. The steps of review process are as follows: ^[4]

Initial screening by editor: An editor evaluates the article to determine if it falls within the scope of the journal and if it meets basic standards for quality. In this stage, the editor may check the text similarity (i.e., text plagiarism) of an article.

Assignment of reviewer: The editor selects qualified experts in the field to review the article from either personal pool of reviewers or by searching on the internet.

Peer review: The reviewers independently evaluate the article, providing written feedback on its strengths and weaknesses of the manuscript and provide constructive comments on how the quality of the presentation may be improved.

Reviewers’ recommendations: The reviewers provide recommendations to the editor, including whether to accept the article as is, reject it, or request revisions. Many of the journals suggest that the reviewers should evaluate the manuscript only and should not provide any recommendations.

Editor’s decision: The editor considers the reviewers’ evaluations and recommendations and makes a final decision on the acceptance, revisions, or rejection of the article.

Author response: The authors receive the reviewers’ evaluations and may be asked to revise the article based on the feedback. In this phase, the authors submit the article again for either reviewers’ re-evaluation or editors’ decision.

Final decision: After considering the authors’ response, the editor makes a final decision on the acceptance, re-review, or rejection of the article.

In this cycle of submission to acceptance, the major role is played by the peer reviewers as the editor may not have enough time to critically evaluate all parts of the manuscript.^[5] In the next section, we discuss about the types of peer review that are currently is in practice.

TYPES OF PEER REVIEW

Peer reviews are commonly categorized according to the identity information of the authors and reviewers. The type may vary according to the publishing houses. The following is the types of peer review currently being practiced:^[6,7]

Single-blind peer review: In single-blind peer review, the identity of the reviewer is kept confidential from the author, but the author’s identity is known to the reviewer.

Double-blind peer review: In double-blind peer review, the identities of both the reviewer and the author are kept confidential from each other.

Open peer review: In open review, both the identity of the reviewer and the author are known to each other, and the review process is often made public, with the reviewer’s comments and the author’s response visible to anyone who has access to the publication.

Along with this classification, the peer review may be categorized according to the time of peer review. Many publishing house and pre-print servers believe that the content of article should be made public first, and then it should get reviewed. Accordingly, the peer review can be divided into two categories as follows:

Pre-publication peer review: In pre-publication peer review, the review process takes place before the publication of the research, with the goal of improving the quality of the research and ensuring that it meets the standards of the field. Till date, majority of the journal follows this model.

Post-publication peer review: In post-publication peer review, the review process takes place after the publication of the research, often through online forums or platforms where other researchers can comment on the work and raise questions or concerns. All the pre-print service providers and some publisher (e.g. F1000 research) follow this model.

Each type of peer review has its own advantages and disadvantages, and the specific type of review used may depend on the policies and preferences of the journal, the type of research being published, and the preferences of the authors and reviewers. Regardless of the type of review, the ultimate goal is to ensure that the research is of high quality and is communicated clearly and accurately to the scientific community. In the next section, we discuss about why researchers need to contribute to the science by conducting peer review.

REASONS TO REVIEW

Contributing to peer review is an important part of the scientific process and the reviewers are doing this from historical time to date aimed to make the science better with their personal capability. However, reviewers review papers without any direct financial gain. There are several reasons why researchers are encouraged to contribute to conduct peer review of research paper in their field. Below are some of the reasons why researchers need to continue their effort to purify science.^[8,9]

Professional development: Reviewing papers provide researchers with an opportunity to stay up-to-date with developments in their field and to improve their own writing and critical thinking skills.

Reciprocal help: Peer reviewing is considered a way for researchers to give back to the scientific community. When a researcher publishes a paper in a journal, it is reviewed by reviewers. Hence, contributing to peer review help get others to get published.

Improving scientific knowledge: Peer review helps researchers identify weaknesses and gaps in a study. This also helps to prepare own research papers with improved presentation.

Enhancing reputation: Researchers who engage in peer review are viewed as active members of the scientific community and are often recognized for their contributions. The review record can now be validated (e.g., web of science researchers profile that acquired the Publons recently) and shown to recruiters or funding agencies for proof of contribution.

Hence, young researchers are encouraged to come forward to take part in the peer review process to make the pool of reviewers always be ready to review. However, novice researchers may find it difficult to find the resources to learn how to peer review. Hence, in the next two sections, first, we enlist the websites from where the authors can learn peer review and then discuss some basic steps of peer review.

WHERE TO LEARN PEER REVIEW

There are several online resources where you can learn about the peer review process and how to be an effective reviewer:^[10]

Publisher websites: Many scientific publishers, such as Elsevier, Springer, and Wiley, offer online resources and guidelines for peer reviewers on their websites.

Online courses: There are several courses online, some are free, and some are paid. The researchers interested to learn the peer review may have a look at the courses listed in [Table 1].^[10] This is not a comprehensive list and is aimed to show some of the available resources that can be accessed freely by any user. We presume, researchers would continue further searches to get more resources.

Blogs and articles: There are many blogs and articles written by experts that offer tips and advice on how to review papers effectively (e.g., https://www.biomedcentral.com/collections/peerreviewtips).

Video tutorials: YouTube has several videos that explain the peer review process and offer tips on how to be an effective reviewer (e.g. https://www.youtube.com/watch?v=LnZd6q-5lg8).

Using these online resources, you can learn the key principles of peer review and gain the skills and knowledge needed to be an effective reviewer. In the following section, we discuss about the basic steps to conduct a review of a research paper.

PROCESS OF PEER REVIEW

The thumb rule for peer reviewing an article is to comment for improvement of the presentation. The reviewers not only detect the errors but also provide constructive comments on how to rectify the errors. Below are the basic steps followed in peer review: ^[11]

Initial screening: The first step in peer reviewing a scientific paper is to determine if the paper falls within your area of expertise and if you have the time and willingness to conduct a comprehensive review. If you are unable to review the paper due to a conflict of interest (e.g., you are affiliated to same institution where the research was conducted) or lack of expertise, it is important to decline the invitation and provide an alternative suggestion for a reviewer if possible.

Preparation for review: After accepting, the journal would send the file directly or provide the login credentials to access the manuscript file. Download the paper or open on browser or take a printout of the manuscript according to preference. Read the guidelines provided by the journal for peer review. Make a quick internet search on relevant topic to be more familiar with the relevant literature.

Conducting the review: Focus on key elements such as the title, research question, methodology, results, and conclusions. Check if the title is framed according to the study design. The abstract should provide a brief account of background, objectives, methods, major finding, and conclusion according to result. Ensure that the keywords are according to MeSH terms. The introduction should be concisely presented with the background of the study. The materials and method section should be arranged in logical flow on how the study was conducted and should be detailed enough to replicate the study. Check if correct statistical methods were used. The result should be checked for completeness and correctness. Ideally, the data of table and figure should not be repeated in the text. Only major findings are written and others are referred to the relevant figure or table. The discussion commonly framed with what was found, the meaning of the finding, comparison with other studies, and limitations. You should evaluate the validity of the conclusions, ensuring that they are supported by the data and no statements made that are beyond the result of the study.

Providing feedback: One of the most important aspects of peer review is providing constructive feedback to the authors. Your review should highlight the strengths and weaknesses of the paper and provide recommendations for improvement. It is important to be specific and constructive in your feedback, avoiding personal comments, and focusing on the scientific merit of the paper. You should also consider any ethical concerns, such as plagiarism or fabrication of data, and address these in your review if necessary.

Making recommendations: These may include accepting the paper as is, requesting revisions, or rejecting the paper. When making your recommendations, it is important to consider the overall contribution of the paper to the field and the potential impact of the research.

Submitting the report: Once you complete your review, it is important to submit to the editor in a timely manner. After submitting the report, the manuscript and associated review report may be deleted from the system.

Peer reviewing a scientific paper is a significant responsibility and requires careful preparation and attention to detail. However, many of the reviewers take too long time to respond.^[12] Always try to think that if you delay reviewing a paper, you have delayed review when you submit a paper to a journal for publication. In the following section, we enlist the common error that a research paper has. This would ease finding the errors and provide constructive comments.

COMMON ERRORS IN RESEARCH PAPER

These are some of the common errors that can be found in research papers.^[13] By recognizing these errors and taking steps to avoid them, researchers can improve the quality and validity of their work and ensure that it is well-received by the scientific community. Reviewers may look for the errors in a research paper.

Inappropriate title: In many cases, the title may seem incomplete or inaccurate or too broad. Title should reflect the whole study in a line.

Poor methodology: This can include a lack of rigor in the study design, inadequate sample size, and inappropriate statistical analysis. Although, the design or sample size may not be changed, statistical errors can be corrected.

Inaccurate or incomplete data: This can occur when data are not collected or recorded correctly, or when important data are omitted. Even some time, the data are presented erroneously during typing.

Inconsistent or incorrect conclusions: This can occur when the conclusions do not match the results or when the conclusions are not supported by the evidence. The conclusion should always adhere to the study.

Lack of relevance or importance: This can occur when the research question or topic is not significant or relevant to the field, or when the results are not novel or interesting.

Poor writing quality: This can include poor grammar, unclear language, or a lack of organization. Unnecessary or redundant text also makes the manuscript poor.

Lack of ethical considerations: This can include the failure to obtain informed consent from participants, the abuse of confidentiality, or the exploitation of vulnerable populations.

Plagiarism: Failure to paraphrase properly the concept, failure to cite relevant literature, or the use of others’ work without proper attribution is all textual plagiarism. Furthermore, figure manipulation and plagiarism are also alarming.^[14,15]

Lack of replication or generalizability: This can occur when the results cannot be replicated by others or when the findings are not generalizable to other populations or settings.

Reviewers may also think checking the article according to the prevalent guidelines (e.g. Strobe, Consort) of reporting so that transparency in scientific reporting is maintained according to guidelines. The following skills may be necessary for a comprehensive peer review.

SKILS NEEDED FOR REVIEW

A person who is asked to peer review a scientific paper needs to have a strong understanding of the field of research and the specific topic covered in the paper. However, it is always to remember that experts also started their first review when they were novice. Below are the desired skills that may be acquired in time and that would help in the process of review.^[16-18]

Familiarity with the relevant literature: The reviewer should have a good understanding of the existing research in the field, as well as any recent developments, to assess whether the paper adds to the current knowledge base.

Critical thinking and analysis: The ability to evaluate critically the validity of the research, the methodology used, and the conclusions drawn.

Ethical considerations: A good understanding of ethical principles in scientific research, including issues related to plagiarism, data fabrication, and conflicts of interest is desired to decide about any ethical deficiency.

Basic statistics: A knowledge of basic statistics is required to check if the used statistical methods are appropriately used or not.

Communication skills: The ability to clearly communicate their comments and suggestions to the authors.

Familiarity with the guidelines of the journal: The reviewer should be aware of the specific guidelines and requirements of the journal, in which the paper will be published, including the scope of the journal and its standards for quality.

Along with these skills, some tools should be kept in handy like a PDF reader with facility to insert sticky notes or highlighting facilities, word processing software that allows inserting comments on a manuscript file, or a note pad where reviewers can write a digital copy of the review reports. In the next section, we highlight some dos and don’ts in a peer review.

DOS AND DONTS IN REVIEW

When writing a peer review report, it is important to maintain a balance between providing constructive criticism and being respectful and professional. Here are some tips that reviewers may do during a peer review.^[19-22]

Accept or reject the review request sent by editor without delay.

Send the review report as soon as possible, not exceeding the stipulated time.

Write a clear and concise summary of the strengths and weaknesses of the paper.

Provide suggestions for improvement of the manuscript.

Write report in a professional and respectful tone, avoiding personal attacks or inflammatory language.

Acknowledge any positive aspects of the paper and areas in which it makes a valuable contribution to the field.

Comment to editor (secret) and authors should corroborate.

Along with this, there are some suggestions on what the reviewers should not act during a peer review.

Do not demand excess explanation that is not needed to add in the manuscript.

Do not criticize the authors’ personal characteristics or backgrounds.

Do not rehash well-known problems in the field, instead focusing on specific issues related to the paper under review.

Disclosing confidential information or making negative comments about the paper to third parties before publication.

Do not stress on grammar (it would be corrected during copy editing). When the reviewers are ready with their skills and expertise, a question may ponder that how editors search for a reviewer. Next section briefly discusses about it.

SEARCHING PEER REVIEWERS

Searching and selecting suitable peer reviewers is perhaps the most difficult task of an editor. Editors typically use a combination of methods to search for peer reviewers for a scientific paper as discussed below:^[23]

In-house databases: Many journals maintain databases of potential reviewers, including those who have reviewed for the journal previously or have indicated their willingness (registered on online journal management system) to review in the future.

Bibliographic databases: Editors may use bibliographic databases such as Web of Science or Scopus to identify potential reviewers based on their publication record in the relevant field.

Colleagues and personal networks: Editors may ask colleagues or other experts in the field for recommendations of potential reviewers.

Open calls for reviewers: Some journals may issue open calls for reviewers on their websites or through professional networks, asking researchers to volunteer to review papers.

From references: Many a times, editors look for similar article in the reference section of a paper and collect email address of corresponding author and approach for review of the paper.

From online avenues: Online resources like JANE (https://jane.biosemantics.org) may be a good source to find suitable expert in the field of research. Furthermore, MeSH on Demand (https://meshb-prev.nlm.nih.gov/MeSHonDemand) also provides related articles while searching keywords from an abstract.^[24] This may also be used for searching reviewers.

The goal of the editor is to identify individuals who are knowledgeable, impartial, and available to review the manuscript within a reasonable timeframe. Along with human reviewer, there is rise of artificial intelligence (AI) that may help conducting a peer review or help completing some components of the review process.^[25] Below are the contributions of AI in the field of peer review of scientific literature.

AI IN PEER REVIEW

Peer review by AI is a relatively new concept in the scientific publishing process. In this type of peer review, AI algorithms are used to replace or assist human reviewers in evaluating the quality of research before it is published.^[26]

AI peer review systems work by analyzing the content of a scientific paper and comparing it to existing research in the field. The AI algorithms use natural language processing and machine learning techniques to identify potential errors, weaknesses, or areas for improvement in the paper. The results of the analysis are, then, presented to a human editor or reviewer, who can use the information to make a decision on whether to accept, reject, or request revisions to the paper.

The potential benefits of AI peer review include increased efficiency, lower costs, and the ability to handle a larger volume of submissions. However, there are also some concerns, such as the potential for bias in the algorithms, the loss of the human element in the review process, and the possibility of incorrect or misleading results.

Some work such as searching text similarity, checking grammar and syntax errors, sentiment analysis, or readability check is done by machines with high accuracy which otherwise is a tough job to do manually by a human. Overall, AI peer review is still in the early stages of development, and its use and impact on the scientific publishing process will likely depend on further research and refinement of the algorithms.

The current peer review system has some advantages and disadvantages.^[27-30] Our personal experience remains good for peer review and they helped us a lot to improve our articles. In contrast, some unusual events also happen. For example, we submitted an article in a journal and even after 9 months, there was no update on the manuscript. We contacted the editor and he replied that as soon as we get the reviewers’ comments, we will provide you the update. Is it logical to keep one manuscript for nine months for review? In following two sections, we discuss about advantages and disadvantages.

ADVANTAGES OF PEER REVIEW

Below are the advantages of the peer review.^[31,32]

Quality: Peer review helps to ensure the quality and credibility of published research by subjecting it to independent evaluation and critique.

Presentation: The peer review process can help authors improve their presentation by providing valuable feedback for improvement.

Validation: Peer review provides an important check on the validity of research findings, helping to identify errors and inconsistencies.

Standard: Peer review helps to maintain standards in a field by ensuring that only the highest quality work is published.

Community: Peer review helps to build a sense of community among researchers by promoting collaboration and sharing of ideas.

DISADVANTAGES OF PEER REVIEW

Some of the disadvantages are listed below.^[33-35]

Time: The peer review process can take a long time. Many reviewers may not respond in stipulated time. Furthermore, they may be busy with other task in their research or academic life to complete the review in time.

Bias: Reviewers may have personal biases that influence their assessments of a piece of work, leading to unfair evaluations.

Number: The pool of qualified reviewers for a given field may be small, and the same individuals may be asked to review multiple submissions.

Objectivity: Reviewers may not be completely impartial, and their subjective opinions can influence the outcome of the review. Moreover, there is no universal checklist for conducting a peer review objectively.

Resistance: Peer review can be resistant to new ideas and innovative research, as reviewers may be more likely to reject submissions that challenge established norms.

Training: Reviewers may not be adequately trained or prepared to assess certain types of submissions, leading to errors in judgment.

Communication: The feedback provided by reviewers may be ambiguous, and authors may not receive sufficient guidance to improve their work.

The current peer review system is including different machines and software to assist in the review process. For example, many journals screen the article for plagiarism, grammatical errors, any gender issues, and undesired words or sentences before beginning the peer review. We presume that in future humans will be more dependent on machines for peer review.

FUTURE OF PEER REVIEW

The future of peer review is likely to involve a combination of traditional methods and new technologies. Even it may be heavily dependent on technology with automatic generation of peer review report.^[36,37] Some of the trends and developments that are likely to shape the future of peer review include:

Technology: The use of technology, such as online platforms and AI, is expected to increase, making the peer review process more efficient and accessible.

Diversity: Efforts to increase the diversity of the reviewer pool are likely to continue, as a more diverse group of reviewers can bring new perspectives and reduce the potential for bias. Due to penetration of internet and computers in remote corner of the world, a researcher from Andaman and Nicober Island, India can review a research paper submitted from United Kingdom.

Pre-publication review: The pre-publication review process may evolve to become more focused on supporting authors in improving their work, rather than just evaluating it for publication. At present, the usage of pre-print servers is increasing day by day.

Collaboration: Due to improved communications and social media use, greater collaboration between researchers and peer reviewers would be possible. Many of the journals now demand name of suggested reviewers. This would increase networking among researchers and potential reviewers.

The future of peer review is likely to be shaped by a combination of technological advancements and changes in the way that research is conducted and communicated, with a continued focus on maintaining the highest standards of quality and integrity.

CONCLUSION

Despite having some drawbacks of peer review, it is still the available best method to maintain sound scientific publication in journals. Novice researchers should get trained for peer review to get included in the current pool of reviewers and start reviewing from the early career stage to reinforce expertise in peer review. This would reciprocally help the researchers in publication of their research papers. While reviewing, reviewers should critically evaluate the manuscript to provide constructive comments with empathy. In future, the peer review process would heavily depend on AI as we presume. However, human resource is always required to check the accuracy of output of machines. Keep reviewing and contribute to the growth of science!