June 18 NISO Virtual Conference: Keynote Speaker: Altmetrics at the Portfolio Level

Paul Groth @pgroth

Web & Media GroupDepartment of Computer ScienceThe Network InstituteVU University Amsterdamhttp://www.few.vu.nl/~pgroth

Altmetrics at the portfolio level

NIH Data Book – (http://report.nih.gov/ndb/index.aspx) Data provided by the Division of Information Services, Reporting Branch

Research Project Grants Applications, awards, and success rates

Metrics: Do metrics matter?Nature 2010http://doi.org/10.1038/465860a

"Outside letters basically trump everything," says Robert Simoni, chairman of the biology department at Stanford University in California.

“Imagine how the academic appointment process might change if search and review committees had access—within an appropriately tagged or linked online CV, for example, or via the ORCID system—to information about the specific contributions made by a candidate to each of his/her works, including contributions that might not otherwise have qualified for ‘authorship’ status?”

Point of view: Faculty appointments and the record of scholarshipAmy Brandhttp://dx.doi.org/10.7554/eLife.00452

Point of view: Faculty appointments and the record of scholarshipAmy Brandhttp://dx.doi.org/10.7554/eLife.00452

Slide 3: http://article-level-metrics.plos.org/files/2013/10/Brand.pptx

Opportunities• Individuals and institutions need better

tools for curating and networking their own record of scholarship

• Institutions need more information about scholarly contribution

• ALMs that reliably differentiate sources of input (general; academic; expert; etc.) would be more useful”

ENTER ALTMETRICS

Altmetrics is the study and use of scholarly impact measures based on activity in online tools and environments.

http://doi.org/10.1371/journal.pone.0048753

http://blog.peerj.com/post/65345738206/changing-the-currency-of-science-to-solve-our-greatest

Thanks Ian Mulvany

ALTMETRICS AS MEASURES OF IMPACT?

“It took approximately a generation (20 years) for bibliographic citation analysis to achieve acceptability as a measure of academic impact."

(Vaughan and Shaw, 2003)

The Research is Happening

htt

p:/

/jaso

npri

em

.org

/self-a

rchiv

ed/5

uni-

post

er.

png

(Birkholtz et al. 2013)

(Fausto et al 2012)

http://ploscollections.org/altmetricshttp://asis.org/Bulletin/Apr-13/AprMay13_Piwowar.html

http://www.cwts.nl/pdf/CWTS-WP-2013-003.pdf

Altmetrics 2014

Zahedi, Zohreh; Fenner, Martin; Costas, Rodrigo (2014): How consistent are altmetrics providers? Study of 1000 PLOS ONE publications using the PLOS ALM, Mendeley and Altmetric.com APIs. figshare. http://dx.doi.org/10.6084/m9.figshare.1041821

http://www.slideshare.net/paulwouters1/issi2013-wg-pw

Bottom Line: use altmetrics as evidence in a larger story

WHAT’S THE LARGER STORY?

DIFFERENT AGGREGATIONS

Aggregations

• Institutional Level– See the next talk

• Faculty• Department• Group

Aggregations

• Institutional Level– See the next talk

• Faculty• Department• Group• Most interesting: adhoc Groups

Projects

• Typical lifetime 2-5 years

• Networks of sub-groups

• Networks of projects• Different measures

of success

29

Published AND discussed AND cited

30

31

32

Events

• Interval lifetime• Changing networks• Virtual teams• Affiliation is transient• Aggregates• Central to scholarly communication

Summary: a broader view

• Almetrics gives a broader view• Altmetrics is must be situated in

context• We should begin looking at the

portfolio level– Multiple aggregations– What cannot be measured

Challenge: Architecture

Thanks

Collaborators: Peter van den Besselaar, Julie Birkholz, Frank van Harmelen, Shenghui Wang, Rinke Hoekstra, Thomas Gurney, Mike Taylor, Anita de Waard, Jason Priem, Dario Taraborelli, Cameron Neylon, Ian Mulvany