UKSG Conference Edinburgh, 12-14 April, 2010

European Research Council

│ 1

UKSG ConferenceEdinburgh, 12-14 April, 2010

Research Assessment in the Humanities and the ESF-ERIH (European Reference

Index for the Humanities) project

Alain PEYRAUBE

CNRS & EHESS, Paris, France

European Research Council

│ 2

ESF-SCH workshop in 2001

We have reached a situation where we can no longer allow ourselves the indulgence of attributing the lack of appropriate tools to a peculiarity of the Humanities and Social Sciences

Specialists in the Humanities need to take into consideration more & more its special characteristics and develop the corresponding tools

It is essential for the future of the Humanities to furnish itself with reliable reference tools, following the example of other scientific disciplines.

European Research Council

│ 3

ESF-SCH workshop in 2001 (2)

• AHCI (ISI-Thomson Scientific Web of Knowledge) is not appropriate

• Urgent need for a European Reference Index for the Humanities (ERIH) as an additional tool for research evaluation, and not the exclusive means

• Request the ESF-SCH to go ahead and try to compile lists of reference journals (Stage 1)

• Develop methodology for including other formats: monographs, book chapters, edited volumes, etc. (Stage 2)

European Research Council

│ 4

ESF-SCH > ERIH Steering Committee

Overall responsibility with the ESF Standing Committee for the Humanities (SCH)

SCH nominated ERIH Steering Committee in Spring 2004, responsible for:

Identification of the disciplinary structure Definition of methodology including the definition of

categories of journals Approval of membership of Expert Panels (members

suggested by MO’s, SCH, ERIH StComm)Validation of journal lists proposed by Expert Panels F. Kiefer (HU), G. Mirdal (DK), A. Mustajoki (FI), A.

Peyraube (FR, Chair), Michael Worton (UK) [Gudrun Gersmann (DE), Marc Waelkens (BE)]

European Research Council

│ 55

ERIH: current disciplinary structure

• Archaeology

• Art and Art History

• Classical Studies

• Anthropology

• Gender Studies

• History

• Hist. & Phil. of Science

• Linguistics

• Literarature

• Musicology

• Oriental & African Studies

• Pedagogical & Educ. Research

• Philosophy

• Psychology

• Religious Studies & Theology

Disciplines under considertation•Archives, Library & Museum Studies•Film, Media & Cultural Studies•Area Studies

European Research Council

│ 6

Peer review: the basis of methodology

• Peer review recognised as the only practicable method of evaluation in basic research (standard method used in evaluation of scientific production) nomination of panel members

• ESF Mos approached – with guidelines – to provide lists of reference journals discipline by discipline

European Research Council

│ 7

ERIH – Journal categories:

A category (expected: 5-20% of all titles):

High-ranking, international level publications;

Very strong reputation among researchers in the field;

Regularly cited all over the world

B category:

Standard, international level publications;

Good reputation among researchers in the field in different countries, occasionally cited all over the world

European Research Council

│ 8

ERIH – Journal categories (2):

• C category:• Important local / regional level publication;• Mainly local / regional readership, but

occasionally cited outside the publishing country

• Only European publications are considered in this group

European Research Council

│ 9

Process of accreditation

• Input: National panels / scientific communities provide lists of journals

• Selection: Expert Panels define scope, analyse and assess input, produce lists

• Consultation: MOs, subject associations (European level and some national), specialist research centres

• Calibration/harmonisation: ERIH Steering Committee; • After approval by ESF-SCH, publication of ‘initial lists’ took

place in 2007• Open feedback process via on-line questionnaire for

editors and publishers• ‘Revised lists’ to be published in 2010

European Research Council

│ 10

Some examples

• Anthropology: 752 journals proposed by Mos 242 journals selected: 40 A (16.5%), 112 B (46.3%), 90 C (37.2%)

• Archaeology: 1312 (Mos) 425: 91 A (21.4%), 170 B (40%), 164 C (38.6%)

• History: 1419 (Mos) 934: 135 A (14.45%), 380 B (40.7%), 419 C (44.9%)

• Linguistics: 1093 (Mos) 585: 96 A (16.4%), 92 B (36.5%), 275 C (47%)

European Research Council

│ 11

Some examples (2)

• Literature: 1453 (Mos) 976: 255 A (26.1%), 481 B (49.2%), 240 C (24.6%)

• Music and musicology: 300 (Mos) 166: 21 A (12.6%), 86 B (51.8%), 59 C (35.5%

• Philosophy: 658 (Mos) 305: 44A (14.4%), 130 B (42.6%), 131 C (42.9%)

• Relig. Stud. &Theology: 745 (Mos) 371: 76 A (20.5%), 203 B (54.7%), 92 C (24.8%)

European Research Council

│ 1212

ERIH: challenges and criticism

• Misunderstandings about the character

of the currently used A/B/C categories: ranking or assessment of audience, distribution and reach?

• Misunderstandings around category C seen as ‘low quality’ when the purpose is to identify quality European journals with, mostly linguistically, limited circulation; identification of quality national journals is the main innovation of ERIH

• Following discussions in the research community, process of renaming ERIH categories is underway

European Research Council

│ 13

More misunderstandings

• Some research councils and research organisations have used ERIH as a tool for assessment of individual research production / productivity

• ERIH “initial lists” were used when they were still under revision

European Research Council

│ 14

New categorization of Journal categories

• National Journals (former category C): European publications with a recognised scholarly significance among researchers in their respective research domains in a particular (mostly linguistically circumscribed) readership group in Europe; occasionally cited outside the publishing country, though their main target group is the domestic academic community

• International Journals (former categories A and B): both European and non-European publications with an internationally recognised scholarly significance among researchers in their respective research domains, and which are regularly cited worldwide

European Research Council

│ 15

New Journal categories (2)

Differentiation between current categories A and B based on a combination of two criteria: influence and scope:

Category A• international publications with high visibility and a very strong

reputation and influence among researchers in the various research domains in different countries

Category B• international publications with significant visibility and a good reputation

and influence in the various research domains in different countries.

European Research Council

│ 16

From 2008 to 2010

• Reorganisation of the expert panels by using a panel rotation mechanism and inclusion of new experts

• Expert Panel meetings revising ‘initial lists’ based on received feedback (adding in new journals, eliminate some journals, changing and revising categories)

• Consolidation of the lists in 2009 (First update) > New lists (Revised lists) to be published in 2010

• Workplan for inclusion of other publications (monographs, edited books, proceedings, etc.)

European Research Council

│ 17

ERIH - contact

www.esf.org/erih

European Research Council

│ 18

Merci

Thank you

谢谢