1 Open Archives Open Archives Meet Meet Open Peer Review? Open Peer Review? Simon Buckingham Simon Buckingham Shum Shum Gary Li Gary Li Knowledge Media Institute Knowledge Media Institute Open University, UK Open University, UK sbs sbs@acm acm.org, [email protected]. .org, [email protected].uk uk Collaborative work with Tamara Sumner (U. Colorado Collaborative work with Tamara Sumner (U. Colorado Boulder, USA) & Mike Wright ( Boulder, USA) & Mike Wright (Unidata Unidata-UCAR,USA) -UCAR,USA) Workshop on The Open Archives Initiative and Peer Review Journals Workshop on The Open Archives Initiative and Peer Review Journals CERN, Geneva, March 2001 < CERN, Geneva, March 2001 < http://documents. http://documents. cern cern . . ch ch / / OAi OAi / /> Overview Overview ■ Just Do It. Then Just Do It. Then Generify Generify It. It. ■ Peer Review Dimensions Peer Review Dimensions ➠ Conversational, Co-Published, Open Peer Review Conversational, Co-Published, Open Peer Review ■ Journal of Interactive Media in Journal of Interactive Media in Education (JIME) Education (JIME) ■ Digital Document Discourse Digital Document Discourse Environment ( Environment (D 3 E) ■ D 3 Emeets an OAI server… meets an OAI server… ■ Future... Future... We have a publication We have a publication problem in HEP. Let’s problem in HEP. Let’s build a solution. build a solution. Publishing/Dissemination Publishing/Dissemination That’s useful! That’s useful! But we don’t know But we don’t know how to do that… how to do that… OAI compliant OAI compliant pub./ pub./ dissem dissem. tools . tools We have a QCC We have a QCC problem in problem in educ educ.tech. .tech. Let’s build a solution. Let’s build a solution. Peer Review Peer Review JIME JIME (process + technology) (process + technology) That’s useful! That’s useful! But we don’t know But we don’t know how to do that… how to do that… D3E + D3E + other generic tools for other generic tools for QCC/interpretation/ QCC/interpretation/ discourse discourse ? ? arXiv arXiv Just Just Do It Do It OAMH protocol OAMH protocol Then Then Generify Generify It It Other fields+ Other fields+ eprint eprint servers… servers… Peer review dimensions Peer review dimensions anonymous anonymous named named appointed appointed open invitation open invitation 1-shot 1-shot conversation conversation reviews discarded reviews discarded reviews preserved reviews preserved reviewers reviewers on their own on their own reviewers reviewers interact interact author right author right of reply of reply author no author no right of reply right of reply Conventional peer review Conventional peer review ■ All peer review models have +/- All peer review models have +/- ■ Anonymous, 1-shot peer review Anonymous, 1-shot peer review ■ Pros Pros + anonymity + anonymity ➠ honesty honesty +1-shot job 1-shot job + “stick with what you know…” + “stick with what you know…” ■ Cons Cons – anonymity – anonymity ➠ lack of accountability lack of accountability – research demonstrates its weaknesses research demonstrates its weaknesses – typically no author right of reply – typically no author right of reply A ‘native internet’ peer review model A ‘native internet’ peer review model Private+Public conversational OPR Private+Public conversational OPR edited + co-published with article … edited + co-published with article … ■ Reviewers Reviewers assigned assigned and and named/ named/ hyperlinked hyperlinked ■ Conversational/argumentation model (web) Conversational/argumentation model (web) ■ Private emails to editor if preferred Private emails to editor if preferred ■ Hybrid 2-step process: private then public Hybrid 2-step process: private then public ■ ➠ revision, publication + open for further revision, publication + open for further comments comments ■ Intellectual trace of the article’s history Intellectual trace of the article’s history
7
Embed
Overview Open Archives Meet Open Peer Review? · Server-side: DiscServer-side: ... PHP Modul e Network Server-side: D33E - -Phorum architecture ... User profiling
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Open ArchivesOpen ArchivesMeetMeet
Open Peer Review?Open Peer Review?Simon Buckingham Simon Buckingham ShumShum
Gary LiGary Li
Knowledge Media InstituteKnowledge Media InstituteOpen University, UKOpen University, UK
Collaborative work with Tamara Sumner (U. ColoradoCollaborative work with Tamara Sumner (U. ColoradoBoulder, USA) & Mike Wright (Boulder, USA) & Mike Wright (UnidataUnidata-UCAR,USA)-UCAR,USA)
Workshop on The Open Archives Initiative and Peer Review JournalsWorkshop on The Open Archives Initiative and Peer Review JournalsCERN, Geneva, March 2001 <CERN, Geneva, March 2001 <http://documents.http://documents.cerncern..chch//OAiOAi//>>
OverviewOverview
�� Just Do It. ThenJust Do It. Then Generify Generify It. It.
�� Peer Review DimensionsPeer Review Dimensions➠➠ Conversational, Co-Published, Open Peer ReviewConversational, Co-Published, Open Peer Review
�� Journal of Interactive Media inJournal of Interactive Media inEducation (JIME)Education (JIME)
�� Digital Document DiscourseDigital Document DiscourseEnvironment (Environment (DD33EE))
�� DD33EEmeets an OAI server…meets an OAI server…
�� Future...Future...
We have a publicationWe have a publicationproblem in HEP. Let’sproblem in HEP. Let’s
build a solution.build a solution.
Publishing/DisseminationPublishing/Dissemination
That’s useful!That’s useful!But we don’t knowBut we don’t knowhow to do that…how to do that…
JIME’sJIME’s peer review lifecycle peer review lifecycle eJournalseJournals: Levels 1-6: Levels 1-6Lancaster, F. W. (1985). The Paperless Society Revisited. American Libraries, 16, (8), 553-555Lancaster, F. W. (1985). The Paperless Society Revisited. American Libraries, 16, (8), 553-555
�� computers used for print productioncomputers used for print production
�� journal distributed in both print andjournal distributed in both print andelectronic formatselectronic formats
�� publication design is rooted in print, butpublication design is rooted in print, butarticles are developed solely for electronicarticles are developed solely for electronicdistributiondistribution
�� interaction between authors and readers isinteraction between authors and readers ispossible; publications can evolve as apossible; publications can evolve as aresult of such interactionsresult of such interactions
�� the inclusion of multimedia contentthe inclusion of multimedia content
�� both interactive participation andboth interactive participation andmultimedia capabilities are supportedmultimedia capabilities are supported
JIMEJIME
�� conversational open peer review conversational open peer review intrinsicintrinsic to tojournal’s review model: journal’s review model: the social contractthe social contract
�� authors encouraged to back claims aboutauthors encouraged to back claims abouttechnology with demonstrations/technology with demonstrations/walkthroughs for readers and reviewerswalkthroughs for readers and reviewers
Interactive demonstration of a CD-ROMInteractive demonstration of a CD-ROM
Readers can ‘play’Readers can ‘play’with thewith the
construction of aconstruction of apainting, aspainting, as
students werestudents wereencouraged to doencouraged to do
execution of aexecution of aprogram usingprogram using
the Java appletthe Java applettested withtested with
studentsstudents
Audio-visual slide presentation, walkingAudio-visual slide presentation, walkingthe reader through a systemthe reader through a system
The authorThe authorintroduces theintroduces the
multimediamultimediasystem with asystem with a
series of slidesseries of slidesand commentaryand commentary
(streaming(streamingaudio)audio)
Video dataVideo data
The authorsThe authorsinclude videoinclude videoclips showingclips showing
their worktheir work(children(children
programming aprogramming arobot)robot)
JIMEJIME
�� conversational open peer review conversational open peer review intrinsicintrinsic to tojournal’s review model: journal’s review model: the social contractthe social contract
�� authors encouraged to back claims aboutauthors encouraged to back claims abouttechnology with demonstrations/technology with demonstrations/walkthroughs for readers and reviewerswalkthroughs for readers and reviewers
�� articles tightly integrated with reviews in aarticles tightly integrated with reviews in aweb document-discussion interfaceweb document-discussion interface
�� at least at least as much feedback as normalas much feedback as normal
�� typically gain an enormous amount fromtypically gain an enormous amount fromdefending against expert peersdefending against expert peers
�� sometimes need to be coaxed intosometimes need to be coaxed intoresponding!responding!
�� …Reviewer’s experience…Reviewer’s experience�� engage in discussions with both authors andengage in discussions with both authors and
reviewersreviewers�� formulate reflective contributions to debatesformulate reflective contributions to debates
in a timely, professional mannerin a timely, professional manner
5
JIME changes… JIME changes… (cont/d)(cont/d)
�� …Reader’s experience…Reader’s experience�� insight into how to interpret the text (insight into how to interpret the text (espesp..
students)students)�� ‘dissenting voices’ are not silenced‘dissenting voices’ are not silenced
�� …the concept of a ‘Publication’…the concept of a ‘Publication’�� multimediamultimedia�� hypertext structures possiblehypertext structures possible�� content can now be distributed across thecontent can now be distributed across the
formal document and the discussion spaceformal document and the discussion space
Conversational Open Peer Review?Conversational Open Peer Review?
�� ProsPros�� Rigorous, accountable quality controlRigorous, accountable quality control
�� At its best can promote interdisciplinary dialogueAt its best can promote interdisciplinary dialogue
�� Reviewers can debate between themselvesReviewers can debate between themselves
�� Works because journal Works because journal PoliciesPolicies and and PracticesPractices have haveevolved with the evolved with the TechnologyTechnology
�� ConsCons�� Both the technology and process are newBoth the technology and process are new
�� More resource intensive for authors and reviewersMore resource intensive for authors and reviewers
�� Better for discursive, multidisciplinary fields?Better for discursive, multidisciplinary fields?
OverviewOverview
�� Peer Review DimensionsPeer Review Dimensions
�� Journal of Interactive Media inJournal of Interactive Media inEducation (JIME)Education (JIME)
�� Hybrid Open Peer Review on the web since 1996Hybrid Open Peer Review on the web since 1996
�� Digital Document DiscourseDigital Document DiscourseEnvironment (D3E)Environment (D3E)
�� From an HTML file to interactive discussionFrom an HTML file to interactive discussiondocument (used to publish JIME)document (used to publish JIME)
�� D3E meets OAID3E meets OAI
�� Future...Future...
DD33E: Digital DocumentE: Digital Document Discourse Environment Discourse Environment
�� From HTML file From HTML file ➠➠ link-rich link-rich document-document-discussion user interfacediscussion user interface
�� Used in many contextsUsed in many contexts
�� Can be used to set up anCan be used to set up an eJournal eJournal using usingany peer review modelany peer review model
D3E-D3E-Phorum Phorum from a data-flow point of view.from a data-flow point of view.The systemThe system’’s main body separates data storage, manipulation ands main body separates data storage, manipulation and
presentation into separate modules.presentation into separate modules.
DD33EE meets an meets an ePrint ePrint server server
generating a D3E discussion spacegenerating a D3E discussion spacefor an archive document…for an archive document…
‘Ubiquitous‘UbiquitousDD33E’E’
...discuss...discussanyany web web
documentdocument
e.g. pass it ane.g. pass it anOAI documentOAI documentURL + specifyURL + specify
thread titlesthread titlesaround which toaround which tofocus commentsfocus comments
and discussionand discussion
Demo: generating a peer reviewDemo: generating a peer reviewdiscussion space for an discussion space for an arXivarXiv document document Adding a comment on anAdding a comment on an arXiv arXiv document document
7
Email confirmation of new review discussion siteEmail confirmation of new review discussion site Future…Future…
�� JIME discourse analysesJIME discourse analyses
�� DD33E-E-OAiOAi integration? integration?
�� depth of technical integration?depth of technical integration?