Workshop How to get peer reviewed Facultat de Biblioteconomia i Documentació, Universitat de Barcelona Barcelona, 20 June 2011 Remedios Melero
May 14, 2015
Workshop
How to get peer reviewed
Facultat de Biblioteconomia i Documentació, Universitat de Barcelona
Barcelona, 20 June 2011
Remedios Melero
Peer review is both a set of mechanisms for evaluating and assuring the quality of research before and after it is funded or published.
It involves subjecting research proposals and draft presentations, papers and other publications to critical evaluation by independent experts (peers).
The reviewers are usually appointed by the funding body or the editors of a journal or other formal channel for communication to which the work has been submitted.
A peer review guide for researchers.
www.rin.ac.uk/peer-review-guide
Scheme from NECOBELAC PORTAL
It is up to the journal editor to decide which system would best ensure scientific integrity (editorial policy) and cooperation. Once a policy is chosen, it should be made clear in the instructions to authors and reviewers.
Comments and data about the effects of open peer review
http://www.bmj.com/content/341/bmj.c5729.full
Conclusion Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ’s website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these
disadvantages.
Scheme from NECOBELAC PORTAL
Peer review does not mean
I like it
Or
I do not like it
Peer review does not mean
I accept it
Or
I do not accept it
Reviewer’s tasks
• Evaluate the content (rigor and originality) of work
• Advise the editor in making decisions about the publication of the work
• Detection of malpractices/missconduct
• The journal needs your scientific expertise, not your editorial assistance
• Some editorial comments are appropriate
• Focus on the science
• The reviewer should consider the appropriateness of the paper for the journal
• The reviewer must also consider whether the paper meets the standards of the journal
What reviewers should focus on:
Extracted from: Ethics of Peer Review: Guide for Manuscript Reviewers Overview by Sara Rockwell (avilable at http://radonc.yale.edu/research/ethics.aspx )
The reviewer is generally ask consider and comment on a variety of issues, including
• The importance and novelty of the work
• The appropriateness of the materials, methods and experimental model systems
• The rigor of the experimental design (including the inclusion of appropriate controls)
• The quality of the data
• The appropriateness of the statistical analyses
• The rigor of the interpretation of the data
• The value of the discussion of the data
• The validity of the conclusions drawn in the paper
The reviewer may also be asked to comment on
• The length of the paper• The writing quality • The clarity, accuracy, and completeness of the figures and tables• The accuracy and adequacy of the introduction which frames the area of the research, of the discussions of prior and related work, and of the citations to the literature
How to select a reviewer…..Adviceable to have criteria for the selection of reviewers
It is recommended that journals monitor the performance of theirreviewers periodically
A checklist for reviewer evaluation will help in choosing more effective and helpful reviewers in the future. Some items that can be included in reviewers’ evaluation are:
• timeliness;• ease of communication;• depth of the review;• clear and instructive comments;• positive attitude;• lack of bias;• willingness to cooperate
Training of reviewers (if possible) may help to increase the quality of the reviewing
The journal should have a review form (guidelines for reviewers) to be sent to the reviewer.
Support to reviewers
The role of editors…briefly
A day in the life of...
a scientific journal editor
Andrew Sugden. Editor of Science
What Andrew asks when considering whether a paper is suitable for Science…
o Does it show something that’s not been shown before? (original)
o Is it surprising; does it show an unexpected result?
o Is it an answer to a long-standing question? Is it significant to other researchers?
o Has everything been done correctly (is it valid)?
o Does it encourage new questions in research?
( Peer review educational resource
http://www.senseaboutscience.net/ )
What happens when the reviewer has looked at it? What do you do then?
I have to consider the reviewers’ comments and make a recommendation about whether the papers should be published in our journal.
Often reviewers don’t agree about a paper, so I need to check that they are being fair and are not too harsh or too kind. If a paper covers more than one area of science, reviewers with different expertise will look at different sections of it. A paper might have both ecology and immunology aspects, for example. Sometimes these reviewers don’t agree. The person reviewing the ecology part of the paper might think it’s excellent but the person reviewing the immunology part might think it’s weak. My job is to check that the reviewers have provided the information to back up their judgements and that they have explained their views. …..
Before I make my final decisions I tell several colleagues about what I plan to do. So now I’m asking another colleague for their point of view on two papers I’d like to accept. ….
Peer review innovation….( see more cases at http://www.alpsp.org/ngen_public/default.asp?id=370 )…………….
http://www.atmospheric-chemistry-and-physics.net/review/index.html
Atmospheric Chemistry and Physics (ACP)
http://jime.open.ac.uk/jime/index
http://www.nature.com/emboj/index.html
Other ways to evaluate scientific papers………….
http://f1000.com
http://www.peerevaluation.org/
Share and get feedback…..
Does peer review detect missconduct?
One of the key aims of peer review is to filter out bad research, including:
• Fabrication of data• Falsification of data• Plagiarism• Failure to disclose conflict of interests and other forms of scientific misconduct
However the peer review process and reviewers themselves are fallible
and
Peer review cannot provide a guarantee against the publication of bad research. In fact there is evidence of the number of retracted papers is rising (Times Higher Education, 2009).
All major publishers have established procedures for handling such cases, and bodies such as the Committee on Publication Ethics (COPE) and the European Association of Science Editors (EASE) provide training and guidance on good practice.
http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=407838
James Parry, acting head of the UK Research Integrity Office (UKRIO), said it was impossible to know for certain the reasons for the increase."It might reflect a real increase in misconduct or, more likely, an increase in detection compared with 20 years ago," he said.
It shows that over nearly 20 years the number of articles produced has doubled, but the number of retractions - still a small fraction of the literature - has increased 20 times. This is equal to a tenfold increase,
factoring in the growth of articles.
Times Higher Education, 2009
Ethics and editors/reviewers/journal staff
Competing interests (also called conflict of interest)
ConfidentialityManuscripts, or parts thereof, or comments of reviewers, must be protected from any personal use byeditors, reviewers or journal staff.
As with authors, it is important for the journal to have clear guidelines to avoid conflicts when articles are reviewed
• Ask reviewers to disclose to the editor any potential competing interests with the paper under review
• Ask reviewers to respect and maintain the confidentiality of the contents of the manuscript, and not to disclose it to anyone without the consent of the editor
• Ask reviewers not to make use of, or quote from, the manuscripts they are reviewing before they are published
• Ask reviewers to obtain written consent of the editor if, for any reason, they want to refer the manuscript to another colleague
• Ask reviewers not to contact the author regarding the manuscript without consulting the editor
• Send manuscripts to at least one foreign or external reviewer when working for a journal that serves a small scientific community, in order to avoid competing interests
Key points to avoid conflict of interests
An alternative to answer some questions about publications ethics…
http://www.publicationethics.org/
The flowcharts are designed to help editors follow COPE’s Code of Conduct and implement its advice when faced with cases of suspected misconduct.