Citation metrics and the stories they tell

International Symposium on the Science of Science���Library of Congress���

March 21st 2016

Citation networks and the stories they tell

Carl T. BergstromUniversity of Washington

Jevin West Martin Rosvall

SciSIP

Jennifer Jacquet Jacob Foster Shelley CorrellMolly King Daril Vilhena Ted BergstromJames Evans Ben Althouse Moritz StefanerDaniel Edler Ian Wesley-Smith Rodney GarrettMichael Jensen Morton Bech Ralph DandreaGregg Gordon

Eigenfactor.org/projects/well-­‐formed/  

Citation is a core institution of ���academic science, tracing the flow of

ideas over time.

The sum of all citations create a vast network of more than a billion citations among more than 100 million papers

Eigenfactor.org/projects/well-­‐formed/  

Every one of those citations represents a careful decision by domain experts.

Eigenfactor.org/projects/well-­‐formed/  

The citation network of science holds a wealth of information about how science

works, and how it can work better.

Eigenfactor.org/projects/well-­‐formed/  

How can we extract ���this information?

Eigenfactor.org/projects/well-­‐formed/  

The first step is to assemble the data. ������

We have compiled citation networks ���from many sources:

���How important is any particular paper, or any particular journal,���

in the network?

Mapequation.org  

Count incoming links���(Impact Factor)

Mapequation.org  

Count incoming links���(Impact Factor)

Use the whole network(Eigenfactor)

Mapequation.org  

Important websites���are linked to by���

important websites.

Important papers���are cited by ���important papers

Important journals���are cited by ���

important journals

Eigenfactor algorithmP = α H + (1 − α ) a.eT

Matrix representing therandom walk over citations Probability of

not teleportingCross-citation Matrixdictating the structureof the citation network

Probability of teleportingto completely new journalweighted by the numberof articles in that journal

EF =100 Hπ[Hπ ]ii∑

Leading eigenvectorof the random walkmatrix P.

Normalization

Bergstrom (2007); West et al (2010)

Applet coding: Daniel EdlerMapequation.org  

The Eigenfactor Algorithm

Study, and publicize, the cost-effectiveness of journal subscriptions

Eigenfactor.org  Bergstrom and Bergstrom 2004 PNAS

Study, and publicize, the cost-effectiveness of open access publishing

Eigenfactor.org  

Ranking authors

“Author-level Eigenfactor performs best in identifying high-impact authors”���

- Dunaiski et al. ��� J. Informetrics May 2016

West et al 2013 JASIST

Ranking articles: ���The Article-Level Eigenfactor (ALEF) Algorithm

Time

Olderpapers

Newer papers

Wesley-Smith et al 2016; West et al in prep.

Image Courtesy of Mark Newman

Small networks reveal structuredirectly.

Dating network in a Michigan high school

Large networks could use some assistance.

Yeast protein interaction network

Ho et al. (2002) Nature

good maps simplify ���and highlight��� relevant structures

Boston MTAGoogle maps

Network community detection ������

We want a modular description of a weighted, directed network: ���

���Most flow on the network occurs within, ���

not between, local modules.

DataCompressing Finding patterns

If we can find a good code for describing flow on a network, we will have solved the dual problem of finding the important structures with respect to that flow.

The map equation tells us the description length for a particular modular structure

The map equation

We conclude that the infomap method by Rosvall and Bergstrom is the best performing… ���

Among other things, the method can be applied to weighted and directed graphs as well, with excellent performances, so it has a large spectrum of potential applications.”

- Lancichinetti and Fortunato (2009)

“

Rosvall and Bergstrom (2008) PNAS

Althouse et al (2009) JASIST

“coverage” Impact factor

Althouse et al (2009) JASIST

1995 2004

1. Determine which structures are statistically significant.���

2. Visualize changes in those structures.

Rosvall and Bergstrom (2010) ���PLoS One

The emergence of neuroscience

The map equation tells us the description length for a particular hierarchical structure

The hierarchical map equation

Rosvall and Bergstrom (2011) ���PLoS One

Rosvall and Bergstrom (2011) PLoS One

Revealing hierarchical structure

Rosvall and Bergstrom (2011) PLoS One

Revealing hierarchical structure

Rosvall and Bergstrom (2011) PLoS One

Revealing hierarchical structure

Rosvall and Bergstrom (2011) PLoS One

Revealing hierarchical structure

Rosvall and Bergstrom (2011) ���PLoS One

Revealing hierarchical structure

Using hierarchical structure for scholarly recommendation

West et al (2016) In press

http://babel.eigenfactor.org  

Using hierarchical structure for scholarly recommendation

1920 1940 1960 1980 2000

0.10

0.15

0.20

0.25

0.30

perc

enta

ge o

f wom

en

What gender disparities still exist across academia?

first author

West et al. 2013 PLoS One

1920 1940 1960 1980 2000

0.10

0.15

0.20

0.25

0.30

perc

enta

ge o

f wom

en

What gender disparities remain ���in scholarly publishing?

last author

first author

West et al. 2013 PLoS One

Eigenfactor.org  

Self-citation rate by gender

●

●

●●●

●●

●

●●●

●●

●●

●●●

●

●●●●

●●●

●●●●

●●●●●●●●●

●●●

●●●●●●●●●●●●●●●

●

●●

●●

●●●

●●●

●●●●●●

●

●●●●●

●●●●●

●●●●●●

●●●●●●●

●●●●●●●●●●●●●

●●●●●●

●●●

●

●

●●

Women's and men's rates of self-citation

���� ���� ���� ���� ���� ��������

���

���

���

���

���

���

����-����� / ����������

Based on > 3 million papers from JSTOR King et al. in prep.

Rates of rates of self-citation do make a difference to impact metrics, particularly the h-index.

– Cameron et al 2016 Bioscience

Self-citations per authorship

”“

King et al. in prep.