SCHOLARLY METRICS BOOTCAMP
Rachel Borchardt and Andrea Michalek
USAIN Preconference
May 4, 2014
What will we cover?
The what and why’s of metrics Bibliometrics overview and discussion Hands-on training with bibliometrics tools Tips for bringing it back home Break Altmetrics overview and discussion Hands-on training with altmetrics tools Bringing it all together
But first..
What are your objectives for the workshop?
What are metrics?
Synonyms/narrower terms: Bibliometrics Scientometrics Impact measures Research metrics Altmetrics
“ … a set of methods to quantitatively analyze academic literature.” (Wikipedia)
Why metrics?
We want to identify high-quality scholarship, but
No one can read everything!
Bibliometrics Impact Quality
Used to evaluate publications, scholars, labs, departments and more
Basic metric categories
Article level Times cited
Journal level Impact Factor SJR, SNIP Google metrics
Author level H-index
(note – this is not a comprehensive list, but covers the most frequently used metrics, and what we’ll be talking about today)
Now, let’s dive into these metrics!
Article level metrics
Article-level Metric: Times Cited Most ubiquitous metric “Basic building block” of bibliometrics Captured by many databases, including
Web of Science Scopus Google Scholar
No one source is definitive and results can vary
Example: Elizabeth Ainsworth Web of Science – 45 documents Scopus – 58 documents Google Scholar – ‘About 17,300 results’
Database/resource coverage will determine times cited as well as number of publications
Lesson: often, an accurate times cited requires an extensive search/comparison
We will explore these resources a bit later!
Web of Science Scopus Google Scholar
Journal-level metrics
Journal-level metric: Impact Factor Developed by Eugene Garfield in 1955 Most widely known and used metric Based on Web of Science citations Distributed exclusively by Journal
Citation Reports (also owned by Thomson Reuters)
2-year and 5-year impact factor metrics
(2-year) Impact Factor Formula
Putting Impact Factor in context Preset disciplines can be chosen and
displayed by impact factor to determine highest impact factor journals
Some high impact factors by discipline
Agricultural Engineering – 4.75, Bioresource Technology
Agriculture, Dairy and Animal Science – 3.494, Genetics Selection Evolution
Agriculture, Multidisciplinary – 2.906, Journal of Agricultural and Food Chemistry
Agricultural Economics and Policy – 2.212, Food Policy
Multidisciplinary Sciences – 38.597, Nature Environmental Studies – 14.472, Nature Climate
Change
Lesson: impact factor varies. Greatly.
So, what’s a ‘good’ impact factor?
An often-asked question with no great answer
Context can help determine this answer Reviewers, particularly outside one’s
discipline, should always have context to avoid misunderstanding impact factor
Discipline specificity remains a major problem with context
Remember: no metric is perfect or absolute!
Let’s explore Journal Citation Reports!
http://library.uvm.edu/research/ Search by Title J, Journal Citation Reports
View a group of journals by subject category or Search for a specific journal
Journal-level metric alternatives to Impact Factor
SJR/SNIP Google Metrics (many others have been proposed, but
in my perspective, are not widely used)
SJR/SNIP
SJR = SCImago Journal Ranking SNIP = Source Normalized Impact per Paper Based on Scopus citations SJR uses an algorithm similar to Google’s to
weigh citations (but otherwise similar to impact factor)
SNIP is based on SJR, but attempts to ‘normalize’ scores between disciplines
Both available at scimagojr.com or via Scopus (Elsevier)
SJR/SNIP Rankings
Similar to Impact Factor, discipline context helps understand SJR/SNIP in comparison
Journal rankings only available on scimagojr.com
Let’s explore SCImago!
http://www.scimagojr.com
This is a freely available resource! (great for libraries/schools with small budgets)
Note that multiple journals can be directly compared in a chart Select “Compare”, then “Journals” Search for up to 4 journals, hit “Compare”,
then select the metric for comparison
Google Scholar Metrics H5-index and H5-median metrics
H5-index = X number of articles in the journal that have been cited X times in the past 5 years H5-index of 10 means that 10 articles in the past 5
years have been cited 10+ times each (remember this – we’ll come back to the H-index
concept!)
H5-median is the median number of citations for all of the articles that fit the H5-index criteria H5-median of 14 means that those 10 articles were
cited a median of 14 times (median = middle number)
Google Scholar rankings
Yet again, different discipline categories! Life Sciences and Earth Sciences
Let’s explore Google Scholar Metrics!
http://scholar.google.com Select “Metrics” along the top Note that a custom list of journals can be
created with a keyword search Try searching for ‘dairy’ or ‘soil’
Author-level metrics
Author-level metric: H-index
Remember H5-index?
H-index is used to measure an author’s output over time
Again, X number of publications that have been cited X or more times
H-index highly dependent on the citation rate of the author’s discipline(s) Just like other measures, context is key Also, like other measures, the metric is only as good as the
source
Where to get H-index?
Web of Science, Scopus, and Google Scholar all have author profiles with an H-index.
However, an incomplete citation / times cited means the H-index is also based on incomplete information.
Let’s practice - Sarah Taylor Lovell Can you find the H-index for Sarah Taylor
Lovell in Web of Science, Scopus and Google Scholar?
Web of Science
To access Web of Science: http://library.uvm.edu/research/ Search by Title W, Web of Science
To find Sarah Taylor Lovell: Author Search in dropdown Lovell, ST Look for matching article – her name is hyperlinked! Clicking on name will pull up (mostly) accurate list
of publications Create Citation Report
Scopus
To access Scopus: http://www.scopus.com
To find Sarah Taylor Lovell: Author Search Lovell, Sarah Taylor Select all relevant results View Citation Overview
Google Scholar
To access Google Scholar: http://scholar.google.com
To find Sarah Taylor Lovell: Search for “Sarah Taylor Lovell” Click on user profile at top
Note: H-index only available for authors with a Google Scholar profile
Lesson: like times cited, an accurate H-index requires extensive search/comparison
Calculating H-index
If citations are incomplete, how do we get an accurate H-index?
First, if they’re close, it may not be worth sweating the details..
Large gaps of coverage between databases may substantiate use of Downloading records from individual
databases Compiling records (usually with Excel) Publish or Perish software
Publish or Perish
Free software program Downloadable at
www.harzing.com/pop.htm Calculates H-index and more Based on Google Scholar data
We won’t explore PoP, but it’s a great tool for downloading/analyzing Google Scholar citations
Bringing metrics back to your institution
What bibliometrics should I recommend to my scholars? Factors to consider
Institutional culture What’s widely used at the institution? What’s the popular perception of metrics?
What do you own / have access to? Resource’s discipline coverage
Social science-oriented scholars may be better served by Scopus; many scientists use Web of Science
Comprehensive results requires more Time Energy Skill
Ways you can help scholars: big picture
Get to know metrics needs at institution Keep an eye on trends in academia and
at your institution Be aware of new products and assess
what you have to ensure it continues to meet user need (so, be a librarian!)
Remember that no metric is perfect, context is key
Ways you can help: concrete activities
Workshops One-on-one consultations Department/lab/admin/etc.
presentations Repository of past works / templates Online research guide (many exist, so
feel free to borrow good ideas!)
Personal plug: Look for my book coming this fall from ACRL Press!
Bibliometrics and altmetrics handbook for librarians
Questions / discussion / hands-on practice