www.library.qut.edu.au LIBRARY SERVICES LIBRARY SERVICES www.library.qut.edu.au D IV IS IO N O F TEC H N O LO G Y , IN FO R M A T IO N A N D LEA R N IN G SUPPORT D IV IS IO N O F TEC H N O LO G Y , IN FO R M A T IO N A N D LEA R N IN G SUPPORT Methodologies for measuring research impact for a new interdisciplinary institute Janet Baker and Dr Nina Prasolova Queensland University of Technology Library
23
Embed
Methodologies for measuring research impact for a new interdisciplinary institute
Methodologies for measuring research impact for a new interdisciplinary institute. Janet Baker and Dr Nina Prasolova Queensland University of Technology Library. Complicated subject and so best told as a story. The request The task – impact report for a “virtual” institute The challenge - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
L I B R A R Y S E R V I C E S
www.library.qut.edu.au
D I V I S I O N O F T E C H N O L O G Y , I N F O R M A T I O N A N D L E A R N I N G S U P P O R TD I V I S I O N O F T E C H N O L O G Y , I N F O R M A T I O N A N D L E A R N I N G S U P P O R T
Methodologies for measuring research impact for a new interdisciplinary institute
Janet Baker and Dr Nina PrasolovaQueensland University of Technology Library
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
Complicated subject and so best told as a story
The request The task – impact report for a “virtual” institute The challenge The negotiations The methodologies The workflow The lessons we learned
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
Characteristics of Institutes or Research Groups
Established institutes
Clear purpose and identity Set agenda, coherent workflow Membership list Physical space with known teams High quality resources already available to measure
research impact Benchmarking progressive and peer group known
L I B R A R Y S E R V I C E SInterdisciplinarity: a field of study that crosses traditional boundaries between academic disciplines, which integrates these different approaches or methods (Center for the Study of Interdisciplinarity).
Boston Interdisciplinary Instituteshttp://www.bu.edu/cgs/citl/interdisciplinary-institutes/
Literature review reveals increasing number of reports and interest in a growing body of literature around issues, policies, challenges regarding IDR
Expectations/ Negotiations: What is the goal?To evaluate performance – factors to consider Look at the traditional metrics – what are you trying to illustrate, what
can you use - Which indicators are consistent for group What tools will give greatest coverage/accuracy Is it possible to analyse cited references - interdisciplinarity Journal subject categories? – WoS and Scopus Percentiles and Average Citation Metrics Percentage share of total publications
Expectations - What is the goal?To predict and for strategic planning Co-authorship graphs – analysis of effective collaborations Co-citation analysis (SciVal) identifying emerging trends Identify Networks outside the discipline boundaries
To improve communications and reporting efficiency Find other research support staff also collecting and
evaluating impact? Tacit knowledge (people evaluating are not the researchers) Knitting together support around the researcher
Percentage of the outputs in the theme Distribution of citations (average and percentile) Collaborations Interdisciplinarity Downloads Attention Metrics Media releases
Search Scopus – using identified research themes then work out citation percentiles
Map QUT output to the total output and citation percentiles
Export and analyse patterns
Assess Collaborations
Assess interdisciplinarity
Workflow
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
Scopus benchmarking (share and percentiles of ‘cited by’)“Tropical Crop*” (All fields) (limit to Country: Australia)
QUT share in the total pubs: 33 (36%) out of 91
3 QUT papers in 10th percentile and9 QUT papers in 25th percentile
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
Author collaborations
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
Analysing collaborations in SciVal
SciVal (Elsevier) January, 2014
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
Summary: what we have learnt
For “virtual” institutes it is important to identify clusters Baselines – sometimes need to change, cultures differ Google Scholar Citations Profile – appears most useful for
some clusters, but data needs to be checked Possible to manually granulate by retrieving those
publications that are in the theme or cluster and then benchmark your institute
Final report needs detailed explanation of metrics and tools chosen and their applicability and replicability
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
What we learned
There is a place for librarians in this exercise
For emerging new institutes librarians could be instrumental providing input into shaping the research clusters and showing future opportunities in interdisciplinarity and collaborations
www.library.qut.edu.au
L I B R A R Y S E R V I C E S
References
Huutoniemi, K., Klein, J. T., Bruun, H., & Hukkinen, J. (2010). Analyzing interdisciplinarity: Typology and indicators. Research Policy, 39(1), 79-88. doi: http://dx.doi.org/10.1016/j.respol.2009.09.011
Roessner, D., Porter, A. L., Nersessian, N. J., & Carley, S. (2013). Validating indicators of interdisciplinarity: linking bibliometric measures to studies of engineering research labs. Scientometrics, 94(2), 439-468.
Wagner, C. S., Roessner, J. D., Bobb, K., Klein, J. T., Boyack, K. W., Keyton, J., . . . Börner, K. (2011). Approaches to understanding and measuring interdisciplinary scientific research (IDR): A review of the literature. Journal of Informetrics, 5(1), 14-26.
Rymer, L. (2011). Go8 Backgrounder 23 | Measuring the impact of research – The context for metric development. Turner, ACT: The Group of Eight. Retrieved from
Center for the Study of Interdisciplinarity. (2013). Retrieved from http://csid.unt.edu/research/ria_overview.html
LSE Public Policy Group. (2011). Maximizing the impacts of your research: A handbook for social scientists (Consultation Draft 3). Retrieved from http://www.lse.ac.uk/government/research/resgroups/LSEPublicPolicy/Docs/LSE_Impact_Handbook_April_2011.pdf