24/09/2004L. Cavazza - Online open peer review/EAHIL Santander 2004 1 Online Open Peer Review: a brand new tool for the evaluation of articles Laura Cavazza.

24/09/2004 L. Cavazza - Online open peer review/EAHIL Santander 2004

1

Online Open Peer Review: Online Open Peer Review: a brand new tool for the evaluation a brand new tool for the evaluation

of articlesof articles

Laura CavazzaLaura CavazzaSoprintendenza beni librari – Regione Emilia-RomagnaSoprintendenza beni librari – Regione Emilia-Romagna

Santander, 9th European Conference of Medical and Health Libraries , Santander, 9th European Conference of Medical and Health Libraries , September 20-25, 2004September 20-25, 2004


2

IntroductionIntroduction


3


4

Peer review: …lights & shadowsPeer review: …lights & shadows


5

DefinitionDefinition

• PEER REVIEW is the revision of articles made by scholars who are peers to the author


6

Peer reviewPeer review


7

Definition Definition (2)(2)

• ’Peer review is an organized procedure carried out by a select committee of professionals in evaluating the performance of other professionals in meeting the standards of their specialty.'

National Library of Medicine (2003) Medical Subject Headings, 2003 Mesh descriptor data, <http://www.nlm.nih.gov/>.


8

Action/functionAction/function

- quality filtre (QC/C)

- guarantee of professional standardsguarantee of professional standards- ‘unrestricted criticism’ (Rennie, 1999)


9

Traditional peer reviewTraditional peer review

Editor

Author/s Referees


10

Pros & consPros & cons

DISADVANTAGES:DISADVANTAGES:

• Peer review is subjective and unreliable • partial (conflict of interest)• biased:

– against innovation– publication (positive results)– statistic– personal-institutional (author)– gender, language, etc.

• slow and expensive• the sum of peers’ mistakes and abuses


11

QuestionsQuestions

Should peer review survive and why?

If so,

what shape should it take?


12

Pros & cons Pros & cons (2)(2)

ADVANTAGES:ADVANTAGES:

• Poor quality research watcher

– (plagiarism, fraud, etc.)

• Quality of article improvement

• Readability of article improvement

• Cost effective/value for money

• ‘Unrestricted criticism’ environment


13

Peer review is essential to the Peer review is essential to the scientific communication processscientific communication process


14

The backgroundThe background


15

The background has The background has recently changedrecently changed

• Technological innovation– Internet

– Computer Supported Cooperative Work


16

The background has The background has recently changed recently changed (2)(2)

• Publishing market– rising costs of journals

– new models of publishing • Open Access Journals and Open Archives


17

The background has recently changed (3)The background has recently changed (3)

Open Access JournalsOpen Access Journals

•Open/free access on the Internet

•copyright lodged with authors

•peer reviewed articles

•authors pay for the publication expenses


18

… … and peer review has and peer review has changed as wellchanged as well

...blind/double blind... ...results blind ...

...bottom-up approach … ...top-down approach...

...internal ... external

...online ...

...open … Online open peer reviewOnline open peer review


19

Open peer reviewOpen peer review

Editor Author/s Referees


20

Online open peer reviewOnline open peer review

• Strong interactivity (CSCW)• Dialogue among author, referees, editor and

readers on the net• Articles linked to peers’ revisions,

editor’s/authors’ comments, different versions, external materials

• Post-publication included in peer review evaluation

• Peer review as a kernel of the journal


21


• Medical J Australia (MJA)

• J Interactive Media Education (JIME)

• Electronic Submission &Peer Review(ESPERE)

• BioMed Central (BMC)

• British Medical J (BMJ)

• Psycholoquy

• Health Library Online(HLO)

Author/s Referee

sss

Readers

Editor

ONLINE JOURNALLinks

(external/internal)

Peer review


22

key pointkey point


Open access journalsOpen access journals


23

A competitive peer review process A competitive peer review process in a open access journal should be:in a open access journal should be:

• Fair/transparent, impartial, reliable

•Able to increase the quality of articles

•Quick and well-timed•Financially sustainable (at least for the author )


24

Pros & consPros & consOnline open peer review and Open access:Online open peer review and Open access:

PROS• Fair, impartial, reliable and

ethical

• Quick and wider dissemination

• Quality improvement of articles and of the process

• Improved article quality through readers’ feedback

CONS• Unfinished validation and

identification of the model in literature

• Costs??

• Possible increase of workload for the journal staff??


25

New scenarios in the future open access New scenarios in the future open access world (1)world (1)

• Articles and documents will change: the official/certified version loses its importance

• medicine: the certified version of the articles must be stated


26

New scenarios in the future open New scenarios in the future open access world (2)access world (2)

• Post-publication evaluation already begins during the peer review process

• IF loses its importance as unique evaluation criterion while/against other systems such as weblogs will be increasing


27

New scenarios in the future open access New scenarios in the future open access world (3)world (3)

• New role of the libraries supporting and financing Open access journals:– paying for authors’/institution publication

expenses– disinvesting in printed and online resources to

the advantages of open access


28

ConclusionsConclusions

The future of scientific The future of scientific communication runs through the communication runs through the

present of peer reviewpresent of peer review


29

ReferencesReferences

Cavazza, L.(2003) Dr Jekyll & Mr Hyde: ruolo, futuro e lati oscuri della peer review nell’editoria biomedica del nuovo millennio (Postgraduate dissertation).

Comba, V. (2000) Comunicare nell’era digitale, Milano, Bibliografica, p. 120-126.

Kassirer, J. P., Campion, E. V. (1994) Peer review. Crude and understudied, but indispensable, JAMA, 1994, p. 96-97.

National Library of Medicine (2003) Medical Subject Headings, 2003 Mesh descriptor data, <http://www.nlm.nih.gov/>.

Nichols, D. (2001) Application of Computer Supported Cooperative Work for libraries, <http://medicina.unica.it/alghero 2001/proceedings/eahil-notes.pdf>

Rennie, D. (2003) Innovation and peer review, <http://www.bmjpg.com/chapter/0727916858_sample.pdf>

Rennie, D. (1999) Editorial peer review: its development and rationale, In Peer review in health sciences (1999) edited by F. Godlee, T. Jefferson, London, BMJ Books, p. 3-13.

Wood, D. (1998) Online peer review?, “Learned Publishing”, 11, 3, p. 193-198.


30

24/09/2004L. Cavazza - Online open peer review/EAHIL Santander 2004 1 Online Open Peer Review: a brand new tool for the evaluation of articles Laura Cavazza.

Documents

definition peer review

open archives

background slide

journal slide

cooperative work slide

quality of articles

net articles

revision of articles