Joerg Bjarne Schlaepfer Doctoral Student KOF Swiss Economic Institute ETH Zuerich Stockholm Nordic Workshop on Bibliometrics and Research Policy 29. October.

Joerg Bjarne SchlaepferDoctoral StudentKOF Swiss Economic InstituteETH Zuerich

StockholmNordic Workshop on Bibliometrics and Research Policy29. October 2013

How Nobel Laureates Would Perform in the Handelsblatt Ranking

Publication counts of Nobel Laureates in economics

2How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Rank Name Award Counts

1 Paul A. Samuelson 1970 106.992 Joseph E. Stiglitz 2001 71.563 Eugene Fama 2013 58.424 Vernon Smith 2002 49.805 Amartya Sen 1998 48.556 James Tobin 1981 48.277 Thomas J. Sargent 2011 47.948 George J. Stigler 1982 47.609 Kenneth J. Arrow 1972 46.7110 Robert M. Solow 1987 46.39

29 October 2013

29 October 2013 3How Nobel Laureates Would Perform in the Handelsblatt Ranking

Doctoral student in economics

At KOF Swiss Economic Institute at ETH Zuerich

Manager of the bibliometric database Forschungsmonitoring

Calculate the bibliometric Handelsblattrankings

Introduction

Introduction Handelsblattranking Nobel Laureates Discussion

Agenda

Introduction

Forschungsmonitoring / Handelsblattranking

Nobel Laureates

Discussion

29 October 2013 4How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Bibliometric webportal forschungsmonitoring.org

5How Nobel Laureates Would Perform in the Handelsblatt Ranking

Forschunsmonitoring.org has been initiated by the Verein für Socialpolitik

Records 6‘0000 (business) economists in Germany, Austria and Switzerland and their 85‘000 publications in peer-reviewed journals

Relational database

Systematic update once a year

29 October 2013

Introduction Handelsblattranking Nobel Laureates Discussion

Login for researchers to validate data

29 October 2013 6How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Handelsblattranking

29 October 2013 7How Nobel Laureates Would Perform in the Handelsblatt Ranking

Almost yearly publication since the year 2005

Economists or business economists

In Germany, Austria and Switzerland

Very well-known and heavily debated

Introduction Handelsblattranking Nobel Laureates Discussion

Methodology of the Handelsblattranking

29 October 2013 8How Nobel Laureates Would Perform in the Handelsblatt Ranking

Research articles in peer-reviewed Journals 1524 Journals from the EconLit Index supplemented by Statistics journals Journal weight

Handelsblatt count depends on Journal weight Number of authors Length of the text is ignored Kind of article: research articles, comments, notes, replys, editorials

Handelsblatt count is the journal weight divided by authors 1 count can be thought of as equivalent to one single-authored research article published

in the American Economic Review (AER)

Sum up the counts of the individual article• Over economist: Individual ranking • Over economists at University: Department ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Pivotal role how the journals are weighted The Handelsblatt-journal weighting list for economics is based on a study by

Pierre-Philippe Combes und Laurent Linnemer (2010): Inferring Missing Citations - A Quantitative Multi-Criteria Ranking of all Journals in Economics

18 September 2012 9Das Handelsblattranking und seine Zeitschriftenliste

Category Weight NumberA+ 1 10A 0.6 26

B+ 0.3 46B 0.2 76

C+ 0.15 112C 0.1 165D 0.05 1089

Introduction Handelsblattranking Nobel Laureates Discussion

Handelsblatt Journal list

A+ JournalsAmerican Economic ReviewEconometricaJournal of FinanceJournal of Financial EconomicsJournal of Monetary EconomicsJournal of Political EconomyNatureQuarterly Journal of EconomicsReview of Economic StudiesScience

29 October 2013 10How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Handelsblattranking 2013 for economists

29 October 2013 11How Nobel Laureates Would Perform in the Handelsblatt Ranking

Rank Name University Age Counts A+ A+, A # Publ.

1 Bruno S. Frey Zeppelin University 72 29 4.08 10.4 290

2 Martin Hellwig Max-Planck Institute 64 23.32 7 19.15 73

3 Roman Inderst Frankfurt 43 21.6 6 16.8 72

4 Oded Stark Klagenfurt - 21.24 1.75 6.75 125

5 Ernst Fehr Zürich 57 20.2611.6

517.2 96

6 Kai A. Konrad Max-Planck Institute 52 18.73 0.75 9.4 117

7 Helmut Lütkepohl Berlin FU 61 17.96 0.33 10.43 95

8 Hans-Werner Sinn ifo Institute 65 16.43 2 7.7 117

9Gerard van den Berg

Mannheim 51 15.94 2.67 13.17 72

10 Enno Mammen Mannheim 58 15.88 1 12.17 84

Introduction Handelsblattranking Nobel Laureates Discussion

Handelsblattranking 2013 for economic departmentsRank 13 Rank 11 University Counts A+ A+, A # Prof. Counts Prof.

1 1 Mannheim 81 17.5 52.4 32 2.2

2 4 Bonn 78 19.7 46.3 26 2.5

3 3 Munich 73 4.6 33.9 27 2.5

4 2Zürich University

71 25.1 49.4 17 3

5 5 Zürich ETH 61 0.8 16.2 10 3.5

6 8 Cologne 53 4.7 24.2 20 2.2

7 6 Frankfurt 49 12 27.3 18 2.4

8 - Lausanne 42 10.9 26.9 14 2.2

8 7 Wien 42 2.3 20.9 14 2.3

10 16 Berlin HU 38 4.5 19.7 13 2.8

29 October 2013 12How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Controversy

29 October 2013 13How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Bibliometrics Handelsblattranking Nobel Laureates Discussion

Rankings are debated heavily and controversly

Convexity of the journal weight plays an important role

Neglects within journal heterogeneity

Provides compact information about research output

Does the responsability lye with the user of the information?

Shift of incentives Holmstrom & Milgrom (1991): If you incentivize tasks that are easy to monitor, people will

put more effort into these tasks.

How Nobel Laureates Would Perform in the Handelsblatt Ranking Matthias Krapf (University of Zuerich) Jörg Schläpfer

Regional and Sectoral Economic Studies Volume 12-3

Abstract

We compile a ranking of the research output of all Nobel Laureates in economics using the Handelsblatt methodology and compare the outcome to the Handelsblatt ranking of economists in the German-speaking area. Our analysis focuses on whether the overall rating scores of the Nobel Laureates are indicative of their high achievements. We discuss the role of the convexity of the employed journal weights and draw conclusions on the Handelsblatt’s methodology.

29 October 2013 14How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Previous studies on Nobel Laureates

Rablen and Oswald (2008) estimate that receiving a Nobel Prize raises life expectancy by between one and two years.

Jones (2010) and Jones and Weinberg (2012) find that innovations that resulted in Nobel Prizes are made increasingly later in life.

Weinberg and Galenson (2005) distinguish between two types of innovators and show that experimental scholars do their most important work later in life than conceptual laureates.

19 October 2012 15How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Publication table of Nobel Laureates in economics

29 October 2013 16How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Rank Name Award Lifetime Pre Award

1 Paul A. Samuelson 1970 106.99 66.752 Joseph E. Stiglitz 2001 71.56 71.563 Eugene Fama 2013 58.42 58.424 Vernon Smith 2002 49.80 45.815 Amartya Sen 1998 48.55 43.936 James Tobin 1981 48.27 37.137 Thomas J. Sargent 2011 47.94 47.948 George J. Stigler 1982 47.60 44.909 Kenneth J. Arrow 1972 46.71 26.1010 Robert M. Solow 1987 46.39 36.43

Observations

Skewed distribution also among Nobel Lareates

Paul Samuelson – an economist with a superior reputation - leads the table by an impressive lead.

15 of the 74 Nobel Laureates in economics earned lower Handelsblatt counts than the top 20 researchers in the German speaking area.

27 Nobel Laureates have earned higher counts than the top researcher in the German-speaking area, who has not won a Nobel Prize. Hamermesh and Pfann (2012): conditional on its impact, the quantity of output has no or

even a negative effect on being awarded honors and prizes in economics.

Our table is not to be understood as a ranking of Nobel Prize winners’ scientific quality. Rather, our goal is to provide examples of different kinds of publication behaviors.

29 October 2013 17How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Publications in A+ and A journals

Two very convex weighting schemes: restricted on A+ and A

Reflecting the tradeoff between publication quantity and publication quality

The correlation between counts from A+ and A journals and total counts is almost perfect with a Spearman coefficient of rank correlation of 0.98

In contrast for the Top of the German speaking area this rank correlation is only 0.63

29 October 2013 18How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Conclusion

For the very top researchers measuring publications in top journals is a good signal for overall output, but this restriction only applies to the very top researchers, i.e. Nobel Laureates.

Using Nobel Laureates as a benchmark is of interest because, as Hirsch (2005) notes, “[f]or the few scientists who earn a Nobel prize, the impact and relevance of their research is unquestionable.” However, any such comparison necessarily involves comparing apples and oranges.

The Handelsblatt ranking is not a perfect measure of past research output.

Standard research ratings that give sufficient weight to quantity are a major source of motivation for aspiring researchers. And it is the level of activity of all economists, which determines how far those can reach who are at the top of the pyramid of the economics community.

29 October 2013 19How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion

Questions / Discussion

29 October 2013 20How Nobel Laureates Would Perform in the Handelsblatt Ranking

Introduction Handelsblattranking Nobel Laureates Discussion