Opening Keynote: Research Metrics, Looking Forward · 2017. 11. 8. · | 0 Andrea Michalek | Managing Director, Plum Analytics and VP Product management, Research Metrics Opening

| 0| 0| 0

Andrea Michalek | Managing Director, Plum Analytics and

VP Product management, Research Metrics

Opening Keynote:Research Metrics, Looking Forward

October 10, 2017

| 1| 1| 1

Why are we talking about metrics?First a question:

| 2| 2| 2

Metrics Drive Decisions

| 3| 3| 3

| 4| 4| 4

| 5| 5| 5

| 6| 6| 6

| 7| 7| 7

Comprehensive Metrics

Give You an Edge

| 8| 8| 8

What are your

institution’s /

country’s

strengths vs. your

peers

What research

areas are getting

funding?

How can we

demonstrate

ROI (Return On

Investment) of

research money?

Who are rising

stars amongst

early career

researchers?

Use metrics to

tell the story of

research

Research Metrics Can Be Used to Identify…

| 9| 9| 9

9

Different Researchers Have Different Needs for Metrics

| 10| 10| 10

Examples of Metrics

Journal Level

• CiteScore

• Journal Impact Factor

• Scimago Journal Rank (SJR)

• Source Normalized Impact Per Paper (SNIP)

Article Level

• Citation Count

• Citations per paper

• Field-Weighted Citation Impact (FWCI)

• Outputs in top quartile

• Citations in policy and medical guidelines

• Usage

• Captures, e.g. bookmarking

• Mentions

• Social media

Researcher Level

• Document Count

• h-Index

| 11| 11| 11

Supporting and developing metrics

for the full lifecycle of research

We strive for broader use of

open metrics that can be

used by everyone

responsibly.

Looking Forward: Research Metrics at Elsevier

11

| 12| 12| 12

Responsible Metrics

• Robustness: basing metrics on the best possible data in terms of

accuracy and scope

• Humility: recognizing that quantitative evaluation should support – but

not supplant – qualitative, expert assessment

• Transparency: keeping data collection and analytical processes

open and transparent, so that those being evaluated can test and verify

the results

• Diversity: accounting for variation by field, and using a variety of

indicators to support diversity across the research system

• Reflexivity: recognizing systemic and potential effects of indicators

and updating them in response

http://www.hefce.ac.uk/pubs/rereports/year/2015/metrictide/

| 13| 13| 13

Elsevier and research metrics

Our Metrics Manifesto

• Need to use different metrics and

common sense

• Decisions should be based on both

quantitative and qualitative input

• Should always use at least two

metrics (more than one way to

‘excellence’)

• The methodologies should be open,

transparent, valid and replicable

• Definitions should be owned by the

community

• Need trust between the parties

using metrics to evaluate

| 14| 14| 14

Innovation in Research Metrics

14

| 15| 15| 15

Research Metrics Throughout the Research Process

15

| 16| 16| 16

Innovation around Societal / Economic Impact

| 17| 17| 17

Represent Metrics for All Types of Research Output

| 18| 18| 18

Metrics to Support

Humanities and Social

Sciences

| 19| 19| 19

• Graduate degrees in psychology

• Half the faculty is actively involved in research

• Books are the main output of faculty members

• Want to showcase faculty work

| 20| 20| 20

Books are their most

common research

artifact type

| 21| 21| 21

Books/Book Chapters Matter

• Citation counts do not do books justice

- Never have & never will

- We need new book metrics

• Important to Social Sciences and Humanities

• Books are often the seminal work from these faculty members

| 22| 22| 22

PlumX Book Specific

Metrics

How many libraries hold this book?

How many Wikipedia articles reference this book?

What are the reviews on Amazon about this book?

What is the book’s average rating?

How many people added this book to their

bookshelf? What reviews have they written? How

did they rate the book?

How many abstract views, downloads, and clicks did

this eBook get?

| 23| 23| 23

Susan Rowland, Ph.D.

Chair of Engaged Humanities & the Creative Life

Pacifica Graduate Institute

Although this book may never

get cited, we still can see

usage, captures, and

mentions.

| 24| 24| 24

PlumX links directly to

Amazon reviews and ratings

| 25| 25| 25

PlumX links directly

to Goodreads

• See reviews and

ratings

• See individuals who

have read or want to

read the book

| 26| 26| 26

Uncover the conversations

the author was not even

aware of.

| 27| 27| 27

Discover how many libraries

hold a copy of the book

| 28| 28| 28

Metrics can help scholars

reclaim their narrative

and tell the story.

| 29| 29| 29

Turn on PlumX Metrics. Integration with Pure Portal

29

The Plum Print on

an article in the

Pure Portal

| 30| 30| 30

Supporting and developing metrics

for the full lifecycle of research

We strive for broader use of

open metrics that can be

used by everyone

responsibly.

Looking Forward: Research Metrics at Elsevier

30

| 31| 31| 31

www.elsevier.com/research-intelligence

Thank you.