University rankings; an overview for the municipality of Delft July 2013

1Challenge the future

International Rankings of Universities

An overview for the municipality of Delft Kim Huijpen, Corporate Policy Affairs | 04/07/13

2Challenge the future

About me

Kim Huijpen• Policy Advisor TU Delft

• Strategic Development / Corporate Policy Affairs

• Member of the Delft city council• Committee Society and Housing • Member of D66

3Challenge the future

International university rankings

1. Context2. Criticism3. Overview4. The position of the TU Delft in important rankings5. New initiatives to improve rankings6. How do we use international rankings?

And how do you use international university rankings?

4Challenge the future

Context

Rankings fill in a need• Stakeholders – students, parents, governments,

accreditation councils, industry (inter)national organizations – want to know the differences between HEI’s and how they perform

Rankings are more and more used (directly or indirectly via reputation)• By the media• By governmental institutions (reallocation of funds)• By students (Asia)• By HEI’s themselves! For marketing purposes or to

select partners for cooperation• By local governments?

5Challenge the future

International rankings, criticism and new developments

Most important international rankings in 2013• QS-, THE-, Shanghai-, Leiden-ranking• Not 4 rankings, but much more (also subject & reputation

rankings)

Criticism• Content: bias for big & old universities, focus on research,

bias for natural & medical sciences, language bias, comparison of whole HEI’s

• Methodology: adding up all kind of indicators, numbering, dubious weighting, intransparency, institutions deliver data, methodological changes

6Challenge the future

Criticism

Conceptual1. Some universities have an advantage: Anglo-Saxon,

beta- and medical disciplines, focus on research, big, old, general

2. You can’t compare whole universities3. You can’t add up all the indicators

Methodology4. Underpinning of the weight factors5. Sensitivity for outliers: best HEI=100 (z-scores are

better)6. Methodological changes in time

Data7. Limited or no insight in the raw data8. Data provided by HEI’s themselves: mistakes,

manipulation

7Challenge the future

Overview: similarities and differences

Ranking

Focus Indicators

Data Time Type

THE ResearchEducationInternat.Income

SubjectiveObjective

Own ResearchDbase (WoS)Data HEI’s

Present GeneralField

QS ResearchEducationInternat.

SubjectiveObjective

Own ResearchDbase (Scopus)Data HEI’s

Present GeneralField

Shangha

i

Research Objective Dbases (e.g. WoS, Nobel-prize.org)

PastPresent

GeneralFieldSubject

Leiden Research Objective Dbase (WoS) Present General

HEEACT Research Objective Dbase (WoS/ESI) Present GeneralFieldSubject

8Challenge the future

TU Delft in rankings ‘12 & spring ‘13

World University Rankings

Engineering/Technology

RankingsOther Rankings

Subject Rankings

Top 10     UIRC Scoreboard (3)

QS Civil & Struct. Eng. (4)

      QS Chemical Eng. (10)

Top 50   QS (18) 

QS Environmental Sciences (17)

    THE (32) QS Materials Science (32)

QS Mechanical Eng. (18)

QS Electrical Eng. (42)

Top 100

THE (77) Shanghai (76-100)

THE Reputation (51-60)

4 QS Subject Rankings

 Top 200 QS (103)     2 QS Subject Rankings

  Leiden (164)

Top 300

Shanghai (201-300)      

Taiwan (276)

9Challenge the future

THE-ranking (with Thomson Reuters)

Fields:• ‘Engineering &

Technology’• ‘Life Sciences’• ‘Clinical, pre-

clinical & Health’

• ‘Physical Science’

• ‘Social Sciences’• ‘Arts &

Humanities’

Ranking by field: based on same 13 indicators with slightly different weights

10Challenge the future

What is citation impact?

• Citation impact is one of the key indicators in most rankings

• With a citation an author acknowledges the original author, year, title, and source of an idea in a new publication

• Citations are measures of the impact of the cited work

• Citation ‘cultures’ differ between disciplines therefore we calculate citation impact normalized for field differences

11Challenge the future

Position of 3TU, LDE & IDEA League in THE ranking

University World University Rankings 2012-13 (2011)

Engineering & Technology Ranking 2012-13 (2011)

TU Delft 77 (104) 32 (22)TU Eindhoven 114 (115) -Universiteit Twente 187 (200) -Universiteit Leiden 64 (79) -Erasmus Universiteit Rotterdam

72 (157) -

Imperial College London 8 (8) 10 (10)ETH Zürich 12 (15) 8 (9)Ecole Polytechnique* 62 (63) 29 (29)Aachen RWTH 154 (168) -* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

12Challenge the future

THE Engineering and Technology Universities 2012

Position THE Engineering & Technology ranking 2012-13 (2011)

1 Caltech, US (2)2 Princeton University, US (3) 2 MIT, US (1)4 University of California, Berkeley,

US (4)5 University of Cambridge, VK (6) /

Stanford University, US (5)Highest non UK/US nr. 8 ETH Zürich (Switzerland)Highest European (non UK/US)

nr. 8 ETH Zürich (Switzerland)

13Challenge the future

Indicators QS ranking

Fields:• Arts & Humanities• Engineering & Technology• Life Sciences & Medicine• Natural Sciences• Social Sciences &

Management

Ranking by field: • Based on same

indicators• Weightings are

different

14Challenge the future

Position of 3TU, LDE & IDEA League in QS ranking

* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

University General ranking (2011)

Engineering and Technology ranking (2011)

Natural Sciences ranking (2011)

TU Delft 103 (104)

18 (18) 91 (79)

TU Eindhoven 158 (146)

67 (61) 186 (177)

Universiteit Twente 224 (226)

101 (116) 267 (229)

Universiteit Leiden 75 (88) - 89 (80)Erasmus Universiteit Rotterdam

99 (103) - -

Imperial College London 6 (6) 6 (6) 11 (11)ETH Zürich 13 (18) 8 (8) 10 (10)Ecole Polytechnique* 41 (36) 36 (36) 43 (40)Aachen RWTH 150

(140)30 (35) 83 (82)

15Challenge the future

Position of TU Delft in QS Subject rankings

16Challenge the future

Indicators Shanghai-ranking (since '03)

Focus Indicators Weighting

Quality of

education

Alumni winning Nobel

prizes and fields medals

Alumn

i

10%

Quality of

faculty

Staff winning Nobel prizes

and fields medals

Award 20%

Highly cited researchers HiCI 20%

Research

output

Articles and papers in

Nature and Science

N&S 20%

Articles and papers in SCI

and SSCI

PUB 20%

17Challenge the future

Position of 3TU, LDE & IDEA League in Shanghai ranking

* ParisTech exists of eleven ‘Grandes Ecoles Paris’ of which Ecole Polytechnique is the most well known.

University Academic Ranking of World Universities - 2012 (2011)

TU Delft 201-300

(151-200)

Universiteit Twente 301-400

(301-400)

TU Eindhoven 301-400

(301-400)

Universiteit Leiden 73 (65)Erasmus Universiteit Rotterdam

151-200

(151-200)

ETH Zürich 23 (23)Imperial College London 24 (24)Ecole Polytechnique* 301-

400(301-400)

RWTH Aachen University 201-300

(201-300)

18Challenge the future

Shanghai-ranking calculated trend

2012 2011 2010 2009 2008 2007 2006 2005 2004 20030

50100150200250300

214 197 185 193 197 194 191234 224 242

Calculated positions TU Delft on Shanghai ranking

These calculated scores are based on calculations per indicator by University of Groningen

19Challenge the future

Leiden ranking

20Challenge the future

Position TU Delft on impact indicators Leiden Ranking 2010-2013

Ranking 2010 2011 2011* 2013 2013*Leiden-ranking (Worldwide top 500)

Old ‘Crown Indicator' (CPP/FCSm)

123      

‘Alternative Crown Indicator' (MNCS)

176 189 99 219 168

Proportion top 10% publications (PPtop 10%)

  179 115 203 164

* Size-independent indicators, impact indicators using fractional counting & exclude publications in special types of journals

21Challenge the future

Position Dutch universities Industry Research Connections 2013

World NL  Uni

PP(UI collab)

1 1 Eindhoven University of Technology 15,6%2 2 Delft University of Technology 14,0%

18 3Wageningen University and Research Centre 10,1%

24 4 University of Twente 9,8%137 5 Erasmus University Rotterdam 7,4%147 6 Leiden University 7,3%153 7 Maastricht University 7,2%160 8 Utrecht University 7,1%186 9 University of Groningen 6,9%214 10 VU University Amsterdam 6,5%253 11 Radboud University Nijmegen 6,1%257 12 University of Amsterdam 6,1%

Or #3 with different counting

22Challenge the future

New developments

Improvement of existing rankings• More attention for education, finance and field (QS,

THE)• More representative survey on reputation (QS, THE)• Rankings per field and subject (Shanghai, QS)

New rankings and classifications (education and third mission, fields and subjects, ranking per indicator, no numbering)• CHE university ranking: BSc-students• CHE excellence ranking: MSc/PhD-students • U-Map (CHEPS): types/profiles• U-Multirank (CHERPA/CHE): institutional and field*

rankings* e.g. engineering

23Challenge the future 2

3

24Challenge the future

What is U-Multirank?

24

Multi-dimensional

• Performance comparison not only on research but also on education, exploitation, international orientation and regional involvement.

Multi-level

• Performance profiles based on a broad set of indicators. The performance profiles are available at two levels: institution as a whole and underlying disciplinary fields.Multi-stakeholder

• Designed in close consultation with stakeholders and intended for - students, administrators, policy makers, employers, etc. - to meet their needs.

Multi-ranking

• Users can decide which areas of performance to include in the comparison of the selected group of universities; in this way U-Multirank produces personalised rankings.

25Challenge the future

Multi-dimensional, -level, -stakeholder, -ranking

25

26Challenge the future

QS Best student cities

Methodology• “Two pre-requisites have been established to identify

the cities evaluated in this exercise. The first is that each city must have a population of over 250,000, the second that it must be home to at least two ranked institutions. Current calculations suggest that 98 cities in the world qualify on this basis.”

1: Paris, 2: London, 3: Boston, 4: Melbourne , 5: Vienna

and 36: Amsterdam

27Challenge the future

Indicators of QS Best student cities

• Student mix• Student Population: as a proportion of the city’s population• International Volume: number of international students in

the city International Ratio: number of international students as a proportion of all students

• Quality of living • Mercer Quality of Living Survey 2011

• Employer activity • Domestic Employer Popularity: Number of domestic

employers who identified one institution in the city as producing excellent graduates

• International Employer Popularity [x2]• Affordability

• Tuition Fees [x2], Big Mac Index & Mercer Cost of Living Index

28Challenge the future

Mercer's Quality of Living ranking

The Quality-of-living index encompasses 39 different factors within the following 10 categories:• Political and social environment• Economic environment• Socio-cultural environment• Medical and health considerations• Schools and education• Public services and transport• Recreation• Consumer goods• Housing• Natural environment

29Challenge the future

How do we use international rankings?

Until now• Participation in

rankings• Internal memos

for the Executive Board

• Annual report• Roadmap 2020 • Website‘facts

and figures’• Marketing and

PR

30Challenge the future

Messages

• More and more international rankings (need)• Are used by several stakeholders and affect your

reputation• Are biased and have methodological drawbacks

• However, methodologies are improving• Nevertheless, important to be in the rankings• It is difficult for specialized universities to reach a high

position in general rankings (TU Delft: technology/engineering)• However, field normalization is improving

• New initiatives to improve international rankings:• U Multirank

31Challenge the future

Questions and discussion:

Which rankings are relevant for the municipality of Delft ?Which rankings do you choose for marketing purposes?

• More information: • www.3tu.nl/uploads/media/Rankings_en_3TU.pdf

• Thanks to Johan Verweij & Jan Salden• I elaborated on Johans presentations & used U-Multirank sheets

of Jan

Kim Huijpen, Policy Advisor, TU Delft / Corporate Policy AffairsT +31 (0)15 27 85296 | E [email protected] | @KimHuijpen

32Challenge the future

References

Some ranking websites:• http://

www.timeshighereducation.co.uk/world-university-rankings/2012-13/world-ranking

• http://www.topuniversities.com/university-rankings/world-university-rankings/2012

• http://www.shanghairanking.com/ARWU2012.html

• http://www.leidenranking.com/ranking

• http://www.topuniversities.com/best-student-cities

• http://nturanking.lis.ntu.edu.tw/DataPage/OverallRanking.aspx

• http://

www.timeshighereducation.co.uk/world-university-rankings/2013/reputation-ranking

Twitter: #unirankings