Www.nifustep.no NIFU STEP Studies in Innovation, Research and Education The Norwegian model and the indicators chosen Gunnar Sivertsen Norwegian Institute.

www.nifustep.noNIFU STEP Studies in Innovation, Research and Education

The Norwegian model

and the indicators chosen

Gunnar Sivertsen

Norwegian Institute for Studies in Innovation, Research, and Education

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

”The dual funding system” (A simplified model excluding international sources, etc.)

Same aims, methods, and effects?

Motivation in Norway: The need for more emphasis on research in the overall funding model for the Higher Education Sector

Budget for HES

Basic: 60% Education: 25% Research: 15%

Strategic Performance based

Scholarly publications

Dissertations

External funding

Since 2006:“Publication points”

“Study points”since 2002

The model in Norway

Panel evaluation, rankings

External funding Peer reviewed

Contract research, etc.

Dissertations

Metric indicators

Publication output

Citations

Patents

Level of analysis:

Individuals

Groups

Programmes

Departments

Institutions

Countries

The publication indicator was developed in 2003-2004 for the Government by the

Norwegian Association of Higher Education Institutions

The Norwegian Higher Education Sector:

The Norwegian Government required:

17,000 researchers…

10,000 scientific and scholarly publications annually…

Collect complete, standardized, verifiable and analyzable data for all publications!

From theology to geology, from internal medicine to international relations, from macroeconomics to nano-technology…

Weight the publications in a comparable way that at the same time stimulates research of good quality!

Seven universities and forty-seven other HE institutions with different missions and subject areas…

Make the sector agree on the model!

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

The standard (so far) database for bibliometric studies

Another starting point: Institutional information systems

Stockholms universitets publikationer

But publishing is a shared activity in shared publication channels

Göteborgs universitets publikationer

This publication comes from Lund, Stockholm and Gothenburg

Another challenge: Sociologisk Forskning is indexed by ISI, but Statsvetenskaplig tidsskrift is not

Publication types in five major fields Based on 33.000 (fractionalized) Norwegian publications from 2005-2008

Coverage of all scientific publications in Norway’s HE sectorBased on 33.000 (fractionalized) publications from 2005-2008

From ISIFrom Norart

One shared national database for all universities. Imported references from bibliographic data sources are validated by researchers or their assistants. Missing references are added.

Books

Principles behind the use of institutional data on a national level

Completeness: All scholarly publications should be included

Simplicity: All scholars can understand the measurement and its effects

Transparency: Every institution can see and check all other institutions’ data. The national database is also online and open to society at large.

Participation: The indicator is developed and maintained in collaboration between the institutions and the authorities

Multiple use of the data: CV’s, applications, evaluations, annual reports, internal administration, bibliography for Open Archives, links to full text, etc.

The next step: Norwegian Science Index from 2011

An open nation-wide database of publications from

The Higher Education Sector

The Institute Sector

The Hospital Sector

Co-authored publications will not be duplicated, but shown as a common publication from two or more institutions

References in the database may lead on to the full texts of publications

Will be used by the Research Council instead of individual publication lists in

Applications

Evaluations

The next step: Norwegian Science Index from 2011

An open nation-wide database of publications from

The Higher Education Sector

The Institute Sector

The Hospital Sector

Co-authored publications will not be duplicated, but shown as a common publication from two or more institutions

References in the database may lead on to the full texts of publications

Will be used by the Research Council instead of individual publication lists in

Applications

Evaluations

Will be part of a new national Current Research Information System in Norway (CRISTIN)

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

Models for performance-based funding of research institutions

Panel evaluation, ranking

Bibliometric models

All disciplines; all publications

Selected disciplines; Citation databases

N

GB

B

DK

S

SF ? AUS

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

The three components of the model

1:Delimitation and documentation

2: Comparable measurement

3: Incentives and funding

A system of ”weigths” takes into consideration different publishing traditions and make them comparable in the same measurement of ”Publication points” at the level of institutions

A definition defines, and the publication database records and validates, structured and standardized bibliographic references to different types of scholarly publications in all fields of research. Author names and addresses are at the same time connected to persons and institutions.

In the overall budget each year, a certain percentage of the basic annual funding from the Government is redistributed between the institutions according to their shares in the total Publication points.

Delimitation of publication data by definition

Definition

A scientific or scholarly publication must:

1. present new insight

2. in a form that allows the research findings to be verified and/or used in new research activity

3. in a language and with a distribution that makes the publication accessible for a relevant audience

4. in a publication channel with peer review*)

*) In addition: Publication channels with authors coming mainly from only one institution are not included.

Three main publication types

Channel

Type

E.g. scientific article in Nature or scholarly ISBN-title on Oxford University Press.

1) Article in ISSN-title2) Article in ISBN-title

3) ISBN-title

Dynamic records of scientific and scholarly ISSN-titles og ISBN-publishers

0001-9887 Africa Today

0001-9909 African Affairs

1062-4783 African American Review

0263-0338 African Archaeological Review

0001-9933 African Arts

1017-6772 African Development Review

0145-2258 African Economic History

1021-3589 African Entomology

1472-5843 African Identities

1684-5315 African Journal of Biotechnology

1021-9730 African Journal of Crop Science

0141-6707 African Journal of Ecology

1438-7890African Journal of Environmental Assessment and Management

1684-5378African Journal of Food, Agriculture, Nutrition

1015-8618 African Journal of Neurological Sciences

0065-4000 African Literature Today

CABI Publishing

Cambridge University Press

Cappelen Akademisk Forlag

Carfax Publishing

Carl Heymanns Verlag

Carlsson bokförlag

Catholic University of America Press

Central European University Press

Channel View Publications

Chinese University Press

Christian Ejlers´s Forlag

18,000 ISSN-titles and 1,000 book publishers so far

All channels can be searched, and new channels can be suggested

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

Dynamic records of scientific and scholarly ISSN-titles og ISBN-publishers

0001-9887 Africa Today

0001-9909 African Affairs

1062-4783 African American Review

0263-0338 African Archaeological Review

0001-9933 African Arts

1017-6772 African Development Review

0145-2258 African Economic History

1021-3589 African Entomology

1472-5843 African Identities

1684-5315 African Journal of Biotechnology

1021-9730 African Journal of Crop Science

0141-6707 African Journal of Ecology

1438-7890African Journal of Environmental Assessment and Management

1684-5378African Journal of Food, Agriculture, Nutrition

1015-8618 African Journal of Neurological Sciences

0065-4000 African Literature Today

CABI Publishing

Cambridge University Press

Cappelen Akademisk Forlag

Carfax Publishing

Carl Heymanns Verlag

Carlsson bokförlag

Catholic University of America Press

Central European University Press

Channel View Publications

Chinese University Press

Christian Ejlers´s Forlag

18,000 ISSN-titles and 1,000 book publishers so far

Publication channels on two levels

Level 2: Higher points to 20 % of the publications

Level 1: Normal points to 80 % of the publications

Level 2 represents the most prestigous channels and is defined in collaboration with the national councils in each field

Differentiated criteria for nomination to level 2

Physics

Economics

Literature

Only most prestigous high-impact journals

Leading journals, series and book publishers with authors from several countries

Publication points

Publication type Level 1 Level 2

Article in ISSN-title 1 3

Article in ISBN-title 0,7 1

ISBN-title 5 8

Publication points are fractionalized if the publication originates from more than one institution

A comparison of the faculties for the Humanities and for Medicine at the University of Oslo (2005)

Institutional research profiles in three different universities (based on publication points 2005-2008)

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

Publication points in Norway’s Higher Education Sector 2004-2009

-1 000

1 000

3 000

5 000

7 000

9 000

11 000

13 000

2004 2005 2006 2007 2008 2009 0 %

5 %

10 %

15 %

20 %

25 %Level 2

Level 1

Level 2 (%)

Shares in the world’s scientific output (ISI 2000-2009)Source: National Science Indicators (NSI), Thomson Reuters

Effects on the institutional level

The institutions have stronger incentives to facilitate research for their researchers

Research is now perceived as a common and institutional responsibility, not only as an individual task

New publications receive attention, not only from external peers, but also internally from the institution

Research management improves with the aid of complete bibliometric information about the research activities

An example of effect at department level: Poster in the toilets of The Centre of Excellence in linguistics at the University of Tromsø

Overview of this presentation

1. Performance based funding

2. Bibliometric datasources

3. An overview of models

4. Definition and delimitation of data

5. Comparable measurement (the publication indicator)

6. The effects so far

7. Conclusions

Electronic information is here to stay

Differences in publication patterns can be recognized

Incomplete representations create tensions within universities

Complete representations support transparence, visibility, and comparability