-
Inheritance patterns in citation networks reveal scientific
memes
Tobias Kuhn,1, Matjaz Perc,2, 3 and Dirk Helbing1, 41Chair of
Sociology, in particular of Modeling and Simulation, ETH Zurich,
8092 Zurich, Switzerland
2Faculty of Natural Sciences and Mathematics, University of
Maribor, Koroska cesta 160, SI-2000 Maribor, Slovenia3CAMTP Center
for Applied Mathematics and Theoretical Physics,
University of Maribor, Krekova 2, SI-2000 Maribor, Slovenia4Risk
Center, ETH Zurich, 8092 Zurich, Switzerland
Memes are the cultural equivalent of genes that spread across
human culture by means of imitation. What makesa meme and what
distinguishes it from other forms of information, however, is still
poorly understood. Here wepropose a simple formula for describing
the characteristic properties of memes in the scientific
literature, whichis based on their frequency of occurrence and the
degree to which they propagate along the citation graph. Theproduct
of the frequency and the propagation degree is the meme score,
which accurately identifies importantand interesting memes within a
scientific field. We use data from close to 50 million publication
records fromthe Web of Science, PubMed Central and the American
Physical Society to demonstrate the effectiveness of ourapproach.
Evaluations relying on human annotators, citation network
randomizations, and comparisons withseveral alternative metrics
confirm that the meme score is highly effective, while requiring no
external resourcesor arbitrary thresholds and filters.
Researchers take delight in meticulously evaluating
scientificoutput and patterns of scientific collaboration. From
citationdistributions [1, 2], coauthorship networks [3] and the
forma-tion of research teams [4, 5], to the ranking of
researchers[68] and the predictability of their success [9] how
wedo science has become a science in its own right. While thenow
famous works of Derek J. de Solla Price [10] and RobertK. Merton
[11] from the mid 1960s has been followed by along run-up towards
maturity and mainstream popularity ofthe field, the rapid progress
made in recent years is largely dueto the increasing availability
of vast amounts of digitized data.Massive publication and citation
databases, also referred to asmetaknowledge [12], along with leaps
of progress in thetheory and modeling of complex systems, fuel
large-scale ex-plorations of the human culture that were
unimaginable evena decade ago [13]. The science of science is
scaling up mas-sively as well, with studies on world citation and
collabora-tion networks [14], the global analysis of the scientific
foodweb [15], and the identification of phylomemetic patterns inthe
evolution of science [16], culminating in the visually com-pelling
atlases of science [17] and knowledge [18].
Science is central to many key pillars of human culture,and
probably the most popular concept to describe the mostinfluential
aspects of our culture is that of a meme. The termmeme was coined
by Richard Dawkins in his book The Self-ish Gene [19], where he
argues that cultural entities such aswords, melodies, recipes, and
ideas evolve similarly as genes,involving replication and mutation
but using human cultureinstead of the gene pool as their medium of
propagation. Re-cent research on memes has enhanced our
understanding ofthe dynamics of the news cycle [20], the tracking
of informa-tion epidemics in blogspace [21], and the political
polarizationon Twitter [22]. It has been shown that the evolution
of memescan be exploited effectively for inferring networks of
diffusionand influence [23], and that information contained in
memes
Electronic address: [email protected]
is evolving as it is being processed collectively in online
socialmedia [24]. The question of how memes compete with eachother
for the limited and fluctuating resource of user attentionhas also
amassed the attention of scientists, who showed thatsocial network
structure is crucial for understanding the di-versity of memes [25]
and that their competition can bring thenetwork at the brink of
criticality [26], where even minute dis-turbances can lead to
avalanches of events that make a certainmeme go viral [27].
While the study of memes in mass media and popular cul-ture has
been based primarily on their aggregated wave-likeoccurrence
patterns, the citation network of scientific liter-ature allows for
more sophisticated and fine-grained analy-ses. Quantum, fission,
graphene, self-organized criticality,and traffic flow are examples
of well-known memes from thefield of physics, but what exactly
makes such memes differentfrom other words and phrases found in the
scientific litera-ture? As an answer to this question, we propose
the followingdefinition that is modeled after Dawkins underlying
defini-tion of the word gene [19]: A scientific meme is a shortunit
of text in a publication that is replicated in citing pub-lications
and thereby distributed around in many copies; themore likely a
certain sequence of words is to be broken apart,altered, or simply
not present in citing publications, the less itqualifies to be
called a meme. Publications that reproducewords or phrases from
cited publications are thus the ana-log to offspring organisms that
inherit genes from their par-ents. In contrast to existing work on
scientific memes, our ap-proach is therefore grounded in the
inheritance mechanismsof memes and not just their accumulated
frequencies. Theabove definition covers memes made up of exact
words andphrases, but the same methods apply just as well to more
ab-stract forms of memes, such as patterns of co-occurrence
andgrammatical structures.
According to our definition, scientific memes are entitiesthat
propagate within the network of citations. To identifythem and
study their properties and dynamics, we thereforeneed databases of
scientific publications that include citationdata. Here we rely on
47.1 million publication records from
arX
iv:1
404.
3757
v1 [
cs.SI
] 14
Apr
2014
-
2WoS. Disciplines:
Natural/Agricultural Sciences
(except Physical Sciences)
Physical Sciences
Engineering and Technology
Medical and Health Sciences
Social Sciences / Humanities
APS. Physical Review Journals:
A: Atomic, molecular, optical phys.
B: Condensed matter, materials phys.
C: Nuclear phys.
D: Particles, fields, gravitation, cosmology
E: Statistical, nonlinear, soft matter phys.
other journals
APS. Selected Memes:
quantum
fission
graphene
self-organized criticality
traffic flow
1FIG. 1: Citation networks of the Web of Science and the
Physical Review datasets reveal community structures that nicely
align with thescientific disciplines and the journals covering
particular subfields of physics. The meme-centric perspective of
the Physical Review citationgraph in the right-hand picture shows
that scientific memes concentrate in relatively isolated
communities of publications that correspond to aparticular subfield
of physics. The generation of the visualizations was based on Gephi
[28] and the OpenOrd plugin [29], which implementsa force-directed
layout algorithm that is able to handle very large graphs. For
details we refer to the network legends and the main text.
the Web of Science, PubMed Central and the American Phys-ical
Society. Due to their representative long-term coverage ofa
specific field of research, we focus mainly on the titles
andabstracts of almost half a million publications of the
PhysicalReview and the pertaining citation data, which were
publishedbetween July 1893 and December 2009. To demonstrate
therobustness of our method, we also present results for the over46
million publications indexed by the Web of Science, andfor the over
0.6 million publications from the open accesssubset of PubMed
Central that covers research mostly fromthe biomedical domain and
mostly from recent years.
The citation graph visualizations presented in Fig. 1 giveus an
intuition about the structure of these networks and thespreading
patterns of scientific memes therein. The leftmostnetwork depicts
the entire giant component of the citationgraph of the Web of
Science database, consisting of more than33 million publications.
It can be observed that different sci-entific disciplines form
relatively compact communities. Thephysical sciences (cyan) are
close to engineering and technol-ogy (magenta) in the top right
corner of the network, but ratherfar from the social sciences and
humanities (green) and themedical and health sciences (red), which
take up the majorityof the left hand side of the network. In
between we have nat-ural and agricultural sciences (blue), which
form an interfacethat connects the other disciplines. Zooming in on
the phys-ical sciences and switching to the dataset from the
AmericanPhysical Society, we get the picture shown in the middle.
Thecolors now encode five Physical Review journals, each cover-ing
a particular subfield of physics. We see that this citationnetwork
has a very complex structure with many small andlarge clusters,
some tightly, while others loosely, connected
with one another. Importantly, even though the employed lay-out
algorithm [29] did not take the scientific disciplines andthe
journal information explicitly into account, the
differentcommunities are clearly inferable in the citation
graphs.
If instead of the Physical Review journals we highlightthe
previously mentioned memes from physics, we obtain therightmost
network presented in Fig. 1. In agreement with ourdefinition of a
scientific meme, we see that most of them ap-pear in publications
that form compact communities in the ci-tation graph. The meme
quantum is widely but by no meansuniformly distributed, pervading
several large clusters. Pub-lications containing the meme fission
form a few connectedclusters limited to area that makes up the
journal Physical Re-view C, which covers nuclear physics.
Similarly, the memesgraphene, self-organized criticality, and
traffic flow (see en-larged area) are each concentrated in their
own medium-sizedor small communities. These points emphasize our
generalmeme-centric perspective that we are employing and
investi-gating for our analysis of the network of scientific
publica-tions.
Results
All words and phrases that occur frequently in the
scientificliterature can be considered important memes, but we
claimthat only the ones that propagate along the citation graph
areactually interesting for a given scientific field. The
importanceof a meme m is thus given by its frequency of
occurrencefm, which is simply the ratio between the number of
publica-tions that carry the meme and the number of all
publications
-
3contained in the evaluated dataset. To quantify the degree
towhich a meme is interesting, we define the propagation scorePm,
which determines the alignment of the occurrences of agiven meme
with the citation graph.
The propagation score Pm is high for memes that fre-quently
appear in publications that cite meme-carrying pub-lications
(sticking) but rarely appear in publications that donot cite a
publication that already contains the meme (spark-ing). Formally,
we define the propagation score for a givenmeme m as its sticking
factor m divided by its sparking fac-tor m. The sticking factor m
quantifies the degree to whicha meme replicates in a publication
that cites a meme-carryingpublication. Concretely, it is defined
as
m =dmmdm
, (1)
where dmm is the number of publications that carry thememe and
cite at least one publication carrying the meme,while dm is the
number of all publications (meme-carryingor not) that cite at least
one publication that carries the meme.Similarly, the sparking
factor m quantifies how often a memeappears in a publication
without being present in any of thecited publications. It is thus
defined as
m =dmmdm
, (2)
where dmm is the number of meme-carrying publicationsthat do not
cite publications that carry the meme, and dmis the number of all
publications (meme-carrying or not) thatdo not cite meme-carrying
publications. For the propagationscore Pm, we thus obtain
Pm =dmmdm
/dmmdm
. (3)
Intuitively, the propagation score compares the
replicationabilities of a meme when its publication is cited with
the ten-dency to appear out of nothing in a publication that does
notcite a meme-carrying publication. Memes with a high propa-gation
score travel mostly along the citation graph.
Having determined the frequency of occurrence fm and
thepropagation score Pm for a particular meme m, we define
theformula for the meme score Mm simply as
Mm = fmPm. (4)
As defined, the meme score has a number of desirable
prop-erties: (i) it can be calculated exactly without the
introductionof arbitrary thresholds, such as a minimal number of
occur-rences, limiting the length of n-grams to consider, or
filteringout words containing special characters; (ii) it does not
de-pend on external resources, such as dictionaries or other
lin-guistic data; (iii) it does not depend on filters, like
stop-wordlists, to remove the most common words and phrases; (iv)
it issimple (we introduce only one parameter; see Methods for
de-tails) and works exceptionally well even on massive datasets;and
(v) it requires virtually no preprocessing of the publica-tion
texts, apart from recommended elementary tokenization
(splitting at blank spaces and detaching trailing
punctuationcharacters) and the transformation to lower case. To
test therobustness and effectiveness of the meme score, we
carefullyevaluate its performance by means of full and
time-preservingrandomization of the citation graphs, by means of
manual an-notation of identified terms, as well as by means of
several al-ternative metrics, including frequency of occurrence,
changesin absolute and relative trends over time, and absolute and
rel-ative differences occurring across journals. We refer to
theMethods section for further details, while here we proceedwith
the presentation of the results obtained with the memescore.
Calculating the meme score for all n-grams in the threedatasets
considered gives us the results presented in Fig. 2.The two
quantities that define the meme score, namely therelatively
frequency and the propagation score, are plottedagainst each other
in the form of heat maps with logarithmicscales. There is no upper
limit to the length of n-grams, andthe presented maps cover without
exception all n-grams witha non-zero meme score. Meme scores are
increasing towardsthe top-right and decreasing towards the
bottom-left corner.Maps A, C and D feature a broad band with a
downward slope,indicating that, in general, more frequent memes
tend to prop-agate less via the citation graph. In the lower half
of eachmap, we see a wedge of very high densities that follows
thelarger band on the bottom-left edge, but getting narrower
to-wards the middle where it ends. Though this wedge has asomewhat
rounder and broader shape for the Web of Science(WoS) database,
overall these patterns look remarkably simi-lar across all datasets
despite their differences with respect totopic, coverage, and size.
This is an indication of universalityin the distribution patterns
of scientific memes. The 99.9%-quantile line (M0.999) is also
surprisingly stable, consideringthat the underlying values range
over five orders of magnitudeor more. Localizing the previously
mentioned physics memesin the APS dataset (map A), we see that they
are located onthe very edge of the top-right side of the band,
where the den-sity of n-grams is very low. Indeed, interesting and
importantmemes are located mostly in this area, which is exactly
whatis reflected by the meme score.
The heat map B in Fig. 2 illustrates a typical case of
whathappens when the APS citation graph is randomized but thetime
ordering of publications is preserved. The number ofterms with a
non-zero meme score decreases dramatically(from 1.4 million in map
A to just 89,356 in map B),the universal distribution pattern of
scientific memes vanishes,and the top-right part where the
top-ranked memes should belocated disappears completely. Naturally,
if the APS citationgraph is randomized without preserving the time
ordering, theoverlap with the original results presented in map A
is evensmaller (not shown). Statistical analysis reveals that
medianvalues of the meme score obtained with the randomized
net-works differ by more than one order of magnitude from
thoseobtained with the original citation graph, with very little
varia-tion between different randomization runs. These results
showthat topology and time structure alone fail to account for
thereported universality in the distribution patterns, and that
thusthe top memes get their high meme scores based on intricate
-
4relative
frequency
102 100 102 104 106106
104
102
100
A APS
n = 1, 372, 365
from titles and abstracts
M0.999 = 0.716
quantum
fissiongraphene
self-organizedcriticality
traffic flow
102 100 102 104 106106
104
102
100
B APS, randomized(time preserving)
n = 89, 356
from titles and abstracts
M0.999 = 0.121
102 100 102 104 106106
104
102
100
C PMC
n = 1, 322, 013
from titles and abstracts
M0.999 = 0.626
102 100 102 104 106 108108
106
104
102
100
D WoS
n = 7, 966, 731
from titles only
M0.999 = 0.560
propagation score
density
ofn-grams:
100
101
102
103
104
105
1FIG. 2: Universality in the distribution patterns of scientific
memes across datasets. Heat maps encode the density of n-grams with
a givenpropagation score and frequency. The meme score increases
towards the top-right and decreases towards the bottom-left corner
in each map.Maps A, C and D, depicting results for publications
from the American Physical Society (APS), the open access subset of
PubMed Central(PMC), and the Web of Science (WoS), respectively,
all feature a broad band with a downward slope, indicating that
more frequent memes tendto propagate less via the citation graph.
The 99.9%-quantile with respect to the meme score distribution
(M0.999) is depicted as a white line.Interesting and important
memes are located mostly around the very edge of the top-right side
of the band (in the vicinity of the 99.9%-quantileline). Heat map B
shows the results obtained with a time-preserving randomization of
the APS citation graph (see Methods for details). Theuniversal
distribution pattern clearly vanishes, thus confirming that the
topology and the time structure of the citation graph alone
cannotexplain the observed patterns, in particular not at the top
end of the meme score distribution.
processes and conventions that underlie the dynamics of
sci-entific progress and the way credit is given to previous
work.
Table I shows the 50 top-ranked memes from the APSdataset, also
indicating their agreement with human annota-tion and whether they
can be found under a subcategory ofphysics in Wikipedia. Several
properties are worth pointingout. First, most of the memes are noun
phrases denoting realand reasonable physics concepts. This is
remarkable giventhat the computation of the simple meme score
formula as-sumes no linguistic knowledge whatsoever and
consideringthat it does not filter out any tokens from the start.
Second,the memes on the list consist of one, two or three
words,which indicates that the meme score does not favor short
orlong phrases, again without applying explicit measures to
bal-ance n-gram lengths. Third, chemical formulas such as MgB2and
CuGeO3 are relatively frequent, which might suggest
thatconventional approaches, filtering such entities out from
thestart, are likely to miss many relevant memes.
In Fig. 3 and Table II, we present results of the
manualannotation of terms identified by meme score as comparedto
randomly selected terms. The general level of agreementbetween the
two annotators is very good, given that the pro-
vided classification is not perfectly clear-cut. In
particular,the agreement is 90% and more for the meme score and
above85% for the random terms. Each of the annotators
consideredaround 86% of the meme score terms to be important
physicsconcepts, agreeing on this in 81% of the cases. With
respectto their linguistic categories, each annotator considered
86%of the meme score terms to be noun phrases, and the two
an-notators agreed on that for 83% of the terms. The
respectivevalues are much lower for the randomly extracted terms.
Only25% (non-weighted) and 19% (weighted) of terms were,
inagreement, found to be important physics concepts, and only33%
(non-weighted) and 25% (weighted) to be noun phrases.The reported
differences between meme score and the tworandom selection methods
are highly significant (p < 1015using Fishers exact test on the
number of agreed classifica-tions). These results confirm that the
meme score stronglyfavors noun phrases and important concepts,
which corrobo-rates its accuracy for the identification of memes in
the scien-tific literature.
Next we compare the meme score to a number of possi-ble
alternative metrics, as defined in the Methods section, andalign
the identified words and phrases with a ground-truth list
-
51. loop quantum cosmology +*2. unparticle +*3. sonoluminescence
+*4. MgB2 +
5. stochastic resonance +*6. carbon nanotubes +*7. NbSe3 +
8. black hole +*9. nanotubes +
10. lattice Boltzmann +*11. dark energy +*
12. Rashba13. CuGeO3 +
14. strange nonchaotic15. in NbSe316. spin Hall +
17. elliptic flow +*18. quantum Hall +*19. CeCoIn5 +
20. inflation +
21. exchange bias +*22. Sr2RuO4 +
23. traffic flow +*24. TiOCl25. key distribution +
26. graphene +*27. NaxCoO2 +
28. the unparticle +
29. black30. electromagnetically induced
transparency +*31. light-induced drift +
32. proton-proton bremsstrahlung +
33. antisymmetrized moleculardynamics +
34. radiative muon capture +
35. Bose-Einstein +
36. C60 +
37. entanglement +
38. inspiral *39. spin Hall effect +*40. PAMELA41. BaFe2As2
+
42. quantum dots +*
43. Bose-Einstein condensates +
44. X(3872) *45. relaxor +
46. blue phases +
47. black holes +*48. PrOs4Sb12 +
49. the Schwinger multichannelmethod +
50. Higgsless +
TABLE I: Top 50 memes with respect to their meme score from the
APS dataset. The symbol + indicates memes where the human
annotatorsagreed that this is an interesting and important physics
concept, while the symbol * indicates memes that are also found on
the list of memesextracted from Wikipedia (see Methods for
details).
of terms extracted from physics-related Wikipedia titles.
Fig-ure 4 summarizes the results, showing that 70% of the top10
memes identified by meme score correspond to terms ex-tracted from
Wikipedia, and 55% of the top 20, 40% ofthe top 50, and 26% of the
top 100. The largest area underthe curve A is obtained for a
controlled noise level = 4 (seeMethods for details), which is
highlighted by the thick blueline. The box plot on the right
compares the outcomes of dif-ferent metrics with respect to A, as
described in the Methodssection. The meme score achieves A-values
that fall com-fortably within the 4050% agreement interval, while
all thealternative metrics score considerably worse, consistently
be-low the 20% agreement baseline. These results indicate thatthe
simple meme score formula performs better than severalalternative
metrics in warranting a reasonably high level ofagreement with the
list of ground-truth memes extracted fromWikipedia.
Having established an accurate meme metric, the dynamicsof memes
and their patterns over time are one of the manythings we can
investigate. As a first step we can track thetemporal changes of
selected memes, as shown in Figure 5for the same five exemplary
memes introduced above. We
physics concept not a physics concept
noun phrase verb adjective or adverb other
meme score
A1A2A1A2
random
A1A2A1A2
weighted random
terms30 60 90 120 150
A1A2A1A2
1FIG. 3: Human annotation agrees with the predictions of the
memescore. The color bars (see legend) encode the level of
agreement withthe meme score list (top) and the two random lists of
terms (middleand bottom). The corresponding statistical analysis
and further de-tails are provided in Table II.
see that these memes have very different histories: fission
wasdominant in the 1960s and 1970s, self-organized criticalityand
traffic flow had their heydays in the 1990s, graphene burstonly
within the last three years of the dataset, while quantumis on a
very long and slow but steady increase without anysignificant
bursts. Looking at the bigger picture, Fig. 6 showsthe top memes
over time, revealing bursty dynamics, akin tothe one reported
previously in humans dynamics [30] and thetemporal distribution of
words [31]. These bursts might be areflection of scientific memes
fast rise and fall with respect tofame and mainstream popularity.
As new scientific paradigmsemerge, the old ones seem to quickly
lose their appeal, andonly a few memes manage to top the rankings
over extendedperiods of time. The bursty dynamics also support the
ideathat both the rise and fall of scientific paradigms is driven
byrobust principles of self-organization [32].
Discussion
By going back to the original analogy to genes put forwardby
Richard Dawkins [19], we propose a definition of scien-
method main class anno- agree- classification p-value for diff.
to:tator ment (agreed) r w
physics concept A1 90.0% 85.3% 81.3%
-
6100 101 102 1030%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
top x terms according to meme score
perc
enta
ge o
f Wiki
pedi
a te
rms
40% of top 50 terms are found on Wikipedia list
0 0.1 0.2 0.3 0.4 0.5
meme score
frequency
maximum absolute change(over time)
maximum relative change(over time)
maximum absolute difference(across journals)
maximum relative difference(across journals)
A (area under curve)
1FIG. 4: The meme score outperforms alternative metrics. The
graph on the left shows the percentage from the x top-ranked terms
accordingto the meme score that also appear on the ground-truth
list of physics terms extracted from Wikipedia, as obtained for
different values ofcontrolled noise (1 10, see Methods for
details). Curves are shown for the individual values of , with the
thick line highlighting thecase = 4, for which the agreement in
terms of the area under the curve A is largest. The box plot on the
right summarizes the quantitativeagreement achieved by the
different metrics (see legend). While the meme score generally
achieves more than 40% agreement, the alternativemetrics all
perform consistently worse, almost exclusively below the 20%
agreement baseline.
0.5 1 1.5 2 2.5 3 3.5 4 4.5x 105
0
5
10
15
publication count
me
me
sco
re (
=
1)
1940
1960
1970
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
quantumfissiongrapheneselforganized criticalitytraffic flow
FIG. 5: The five exemplary memes exhibit very different
histories in terms of their meme scores. Four of them show bursts
at different pointsin time, while the fifth quantum shows a very
steady and almost linear path. The time axis is scaled by
publication count.
tific memes based on their inheritance patterns on the
citationgraph of publications. We present the meme score, a met-ric
to identify scientific memes, defined as the product of
thefrequency of occurrence and the propagation score, wherebythe
latter determines the degree to which the occurrence of ameme is
aligned with the citation graph.
We have shown that the meme score can be calculated ex-actly
without the introduction of arbitrary thresholds or fil-ters,
without the usage of external resources such as dictionar-ies, and
without noteworthy preprocessing of the publicationtexts. The
method is fast and reliable, and it can be appliedon massive
databases. We have demonstrated the effective-ness of the meme
score on more than 47.1 million publicationrecords from the Web of
Science, PubMed Central, and theAmerican Physical Society.
Moreover, we have evaluated theperformance of the proposed meme
score by means of fulland time-preserving randomization of the
citation graphs, by
means of manual annotation of publications, as well as bymeans
of several alternative metrics. We have provided sta-tistical
evidence for the agreement between human annotatorsand the
meme-score results, and we have shown that it is su-perior to
alternative metrics. We have also confirmed that theobserved
patterns cannot be explained by topological or tem-poral features
alone, but are grounded in more intricate pro-cesses that determine
the dynamics of the scientific progressand the way credit is given
to preceding publications. The top-ranking scientific memes reveal
bursty time dynamics, whichmight be a reflection of the fierce
competition of memes forthe limited and fluctuating resource of
scientists attention.
-
70.5 1 1.5 2 2.5 3 3.5 4 4.5
x 105
0
2
4
6
8
10
12
publication count
mem
e sc
ore
1940
1960
1970
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
grapheneentanglement
MgB2
nanotubescarbon nanotubes
quarkneutrino
BoseEinsteinquantum Hall
blackC
60Hubbard model
quantum wellsgraphite
reactionsphotoemission
black holetricritical
Kondosuperconducting
fissionMeV
diffuse scattering
FIG. 6: Time history of top physics memes based on their meme
scores obtained from the American Physical Society dataset. The
time axisis scaled by publication count. Bars and labels are shown
for all memes that top the rankings for at least ten out of the
displayed 911 pointsin time. The gray area represents the
second-ranked meme at the given time. The bursty dynamics seem to
indicate that both rise and fall arefast, and that for the majority
of scientific memes the popularity is fleeting.
Methods
Controlled noise and discounting free-riding
The effectiveness of the propagation score, as defined inEq. 3,
can be further improved by adding a small amount ofcontrolled noise
, thus obtaining
Pm =dmmdm +
/dmm + dm +
. (5)
The controlled noise corrects for the fact that any of the
fourbasic terms can be zero, and it also prevents that phrases
witha very low frequency get a high score by chance. To
illustratethe latter, consider a publication that is cited only
once, whilethe citing publication is never cited. If these two
publicationshappen to share a phrase that does not exist otherwise,
forexample because the second publication reproduces a
shortsequence of the text of the first, then this phrase would
getthe maximum propagation score (it sparks once and then italways
sticks). The controlled noise corresponds to fictitiouspublications
that carry all memes and cite none, plus another publications that
carry no memes and cite all. This decreasesthe sticking factors and
increases the sparking factors for allmemes, thereby reducing all
meme scores; very slightly sofor frequent memes but heavily for
rare ones. Our tests show
that a small amount of noise (e.g. = 3 as used throughoutthis
work unless stated otherwise) are sufficient to solve
theabove-mentioned problem.
Another matter that deserves attention is the potential
free-riding of shorter memes on longer ones. Memes can be partof
larger memes, and it pays to check whether a given memesticks on
its own or just free-rides on the popularity of alarger meme. For
example, consider the multi-token memethe littlest Higgs model,
which contains the specific tokenlittlest that rarely occurs
otherwise. The meme littlesttherefore gets about the same
propagation score as the longmeme. Yet the larger meme is clearly
more interesting, andwe should thus discount for sticking behavior
that is due onlyto the free-riding on larger memes. This can be
achieved byredefining the term dmm in Eq. 1 to exclude
publicationswhere the given meme appears in the publication and its
citedpublications only within the same larger meme. If littlest,for
example, is always followed by Higgs in a given publi-cation and
all its cited publications, then this publication shallnot
contribute to the dmm term for m = littlest.
-
8Graph randomization
We use graph randomizations to verify that the reported re-sults
are not rooted in generic properties of the examined net-work, i.e.
we want to rule out that the observed effects can beexplained by
network topology and chance alone. For that weuse networks that
have exactly the same topology as the origi-nal one but where the
article texts (i.e. titles and abstracts withtheir memes) are
randomly assigned to the nodes. Each nodetherefore owns it position
in the network to one particular pub-lication but has text attached
that comes from a different one.This kind of randomization,
however, still leaves room for theunlikely possibility that the
time order of the publications hasa major effect on the meme score,
in particular the simplefact that citations go only backwards in
time. To rule thisout, we also perform time-preserving
randomizations of thecitation network, shuffling only publications
that were pub-lished within narrow consecutive time windows.
Concretelywe use time windows of 1000 publications, meaning that
after shuffling no publication has moved more than 1000positions
forward or backward from the original chronologi-cal order.
Human annotation
To test whether human annotators confirm that phrases witha high
meme score are indeed interesting and important con-cepts in the
respective scientific field such as physics, we de-fine the
following two categories for manual annotation: (i)the phrase is
not a meaningful term or not an important con-cept of physics, and
(ii) the phrase is an important concept orentity of physics it
could appear as the title of an entry ofa comprehensive
encyclopedia on physics. Our hypothesis isthat the phrases with a
high meme score would tend to endup in the second category, while
the phrases with a low memescore would end up in the first. In
addition, we asked our an-notators to determine the linguistic
categories of the phrases,for which we defined the following
classes: (i) noun phrase,(ii) verb, (iii) adjective or adverb, and
(iv) other. Our intu-itive assumption is that memes would mostly
have the form ofnoun phrases.
The set of phrases used for this evaluation consisted of thetop
150 memes with respect to their meme score, extractedfrom the
American Physical Society dataset, plus another twosets for
comparison of 150 randomly drawn phrases each. Forthe two
comparison sets, we have considered all phrases thatappear in at
least 100 publications. From these, 150 termswere drawn randomly
without taking into account their fre-quency, i.e., frequent terms
had the same chance of being se-lected as infrequent ones, whereas
the second 150 terms weredrawn with a weight that corresponded to
their frequency, i.e.,a term appearing 10000 times was ten times
more likely to beselected than a term appearing 1000 times.
Moreover, to ruleout effects of different n-gram lengths, we made
sure that thetwo batches of random terms followed exactly the same
lengthdistribution as the main sample extracted based on the
memescore. The resulting 450 terms were shuffled and given to
two
human annotators, both PhD students with a degree in physics,who
independently annotated each of the terms according tothe two
criteria (physics concept and linguistic category).
Alternative metrics
We have used the following metrics as alternatives to thememe
score: (i) frequency the most frequent terms (upto 3-grams)
skipping the first x terms to filter out generalwords like of and
method (setting x in the range from 0 to500); (ii) maximum absolute
change over time the highest-scoring terms (up to 3-grams,
occurring in at least 10 publica-tions) with respect to maximum
absolute change in frequencyfrom one time window of x publications
to the next on chrono-logically sorted publications (we set x in
the range from 1000to 100,000); (iii) maximum relative change over
time thesame as (ii) but based on relative changes; (iv) maximum
ab-solute difference across journals the highest-scoring terms(up
to 3-grams, occurring in at least 10 publications) with re-spect to
maximum absolute difference in relative frequencyfrom one journal
to another, considering only journals withat least x publications
and optionally excluding the old jour-nals Physical Review Series I
and II (we set x to 0, 1000, and10,000); (v) maximum relative
difference across journals the same as (iv) but based on relative
changes.
Metric (i) is based on the assumption that important memesare
frequent but not as frequent as the small class of generalwords
that can be found in all types of texts. Metrics (ii) and(iii) are
based on an idea proposed in [32], being that interest-ing memes
exhibit trends over time. Words like approachand the might have a
very high frequency, but they are notsubject to strong trends over
time, as compared to terms suchas graphene. Metrics (iv) and (v)
are based on the intuitionthat phrases occurring mostly in specific
journals but not inothers must be specific concepts of the
particular field of re-search.
To compare these metrics, we need to establish some sortof
ground-truth list of memes to compare the extracted termsagainst.
For that, we automatically extracted 5178 terms fromWikipedia. We
collected the titles of all articles and termsredirecting to them
from the categories physics, appliedand interdisciplinary physics,
theoretical physics, emerg-ing technologies, and their direct
sub-categories, but filter-ing out terms that appear in less than
10 publications of theAmerican Physical Society dataset. While it
is unreasonableto expect that any metric would be able to perfectly
reproducesuch a Wikipedia-based list of memes, a considerable
overlapshould nevertheless be achievable for good metrics.
To quantify the agreement between the top memes identi-fied by a
particular metric and the Wikipedia list, we use thenormalized area
A under the curve as shown on the left ofFigure 4. The step-shaped
curved has a log-scaled x-axis run-ning up to the number of terms s
on the ground-truth meme list(s = 5178 in our case) and the y-axis
running from 0 (no over-lap) to 1 (perfect overlap). Limiting cases
are A = 1, repre-senting perfect agreement, and A = 0, representing
no agree-ment between the two compared lists. Values 0 < A <
1 rep-
-
9resent partial agreement, giving an agreement between
higher-ranked memes more weight than to an agreement
betweenlower-ranked ones.
Acknowledgments
This research was supported by the European Commissionthrough
the ERC Advanced Investigator Grant Momentum
(Grant No. 324247) and by the Slovenian Research Agencythrough
the Program P5-0027. In addition, we would like tothank Karsten
Donnay, Matthias Leiss, Christian Schulz, andOlivia Woolley-Meza
for their help.
[1] Redner, S. How popular is your paper? an empirical study
ofthe citation distribution. Eur. Phys. J. B 4, 131134 (1998).
[2] Radicchi, F., Fortunato, S., and Castellano, C. Universality
ofcitation distributions: Toward an objective measure of
scientificimpact. Proc. Natl. Acad. Sci. USA 105, 1726817272
(2008).
[3] Newman, M. E. J. Coauthorship networks and patterns of
scien-tific collaboration. Proc. Natl. Acad. Sci. USA 101,
52005205(2004).
[4] Guimera`, R., Uzzi, B., Spiro, J., and Amaral, L. A. N. Team
as-sembly mechanisms determine collaboration network structureand
team performance. Science 308, 697702 (2005).
[5] Milojevic, S. Principles of scientific research team
formationand evolution. Proc. Natl. Acad. Sci. USA 111,
39843989(2014).
[6] Hirsch, J. E. An index to quantify an individuals
scientificresearch output. Proc. Natl. Acad. Sci. USA 104, 16569
(2005).
[7] Radicchi, F., Fortunato, S., Markines, B., and Vespignani,
A.Diffusion of scientific credits and the ranking of
scientists.Phys. Rev. E 80, 056103 (2009).
[8] Petersen, A. M., Wang, F., and Stanley, H. E. Methods for
mea-suring the citations and productivity of scientists across
timeand discipline. Phys. Rev. E 81, 036114 (2010).
[9] Penner, O., Pan, R. K., Petersen, A. M., Kaski, K., and
Fortu-nato, S. On the predictability of future impact in science.
Sci.Rep. 3, 3052 (2013).
[10] de Solla Price, D. J. Networks of scientific papers.
Science 149,510515 (1965).
[11] Merton, R. K. The matthew effect in science. Science
159,5363 (1968).
[12] Evans, J. A. and Foster, J. G. Metaknowledge. Science
331,721725 (2011).
[13] Michel, J. B., Shen, Y. K., Presser Aiden, A., Veres, A.,
Gray,M. K., The Google Books Team, Pickett, J. P., Hoiberg,
D.,Clancy, D., Norvig, P., Orwant, J., Pinker, S., Nowak, M. A.,and
Lieberman Aiden, E. Quantitative analysis of culture usingmillions
of digitized books. Science 331, 176182 (2011).
[14] Pan, R. K., Kaski, K., and Fortunato, S. World citation
andcollaboration networks: uncovering the role of geography
inscience. Sci. Rep. 2, 902 (2012).
[15] Mazloumian, A., Helbing, D., Lozano, S., Light, R. P.,
andBorner, K. Global multi-level analysis of the scientific
foodweb. Sci. Rep. 3, 1167 (2013).
[16] Chavalarias, D. and Cointet, J.-P. Phylomemetic patterns
inscience evolution the rise and fall of scientific fields. PLoSONE
8, e54847 (2013).
[17] Borner, K. Atlas of Science. MIT Press, Cambridge,
MA,(2010).
[18] Borner, K. Atlas of Knowledge. MIT Press, Cambridge,
MA,(2014).
[19] Dawkins, R. The Selfish Gene. Oxford University Press,
Ox-ford, (1989).
[20] Leskovec, J., Backstrom, L., and Kleinberg, J.
Meme-trackingand the dynamics of the news cycle. In Proceedings of
ACMSIGKDD, 497506, (2009).
[21] Adar, E. and Adamic, L. A. Tracking information epidemicsin
blogspace. In Proceedings of IEEE/WIC/ACM, 207214,(2005).
[22] Conover, M., Ratkiewicz, J., Francisco, M., Goncalves,
B.,Menczer, F., and Flammini, A. Political polarization on
Twitter.In Proceedings of ICWSM, 8996, (2011).
[23] Gomez Rodriguez, M., Leskovec, J., and Krause, A.
Inferringnetworks of diffusion and influence. In Proceedings of
ACMSIGKDD, 10191028, (2010).
[24] Simmons, M. P., Adamic, L. A., and Adar, E. Memes
online:Extracted, subtracted, injected, and recollected. In
Proceedingsof ICWSM, 353360, (2011).
[25] Weng, L., Flammini, A., Vespignani, A., and Menczer, F.
Com-petition among memes in a world with limited attention.
Scien-tific Reports 2 (2012).
[26] Stanley, H. E. Introduction to Phase Transitions and
CriticalPhenomena. Clarendon Press, Oxford, (1971).
[27] Gleeson, J. P., Ward, J. A., OSullivan, K. P., and Lee, W.
T.Competition-induced criticality in a model of meme
popularity.Phys. Rev. Lett. 112, 048701 (2014).
[28] Bastian, M., Heymann, S., and Jacomy, M. Gephi: an
opensource software for exploring and manipulating networks.
InICWSM, 361362, (2009).
[29] Martin, S., Brown, W. M., Klavans, R., and Boyack, K.
W.OpenOrd: an open-source toolbox for large graph layout.
InIS&T/SPIE Electronic Imaging. International Society for
Op-tics and Photonics, (2011).
[30] Barabasi, A. L. The origin of bursts and heavy tails in
humansdynamics. Nature 435, 207211 (2005).
[31] Altmann, E. G., Pierrehumbert, J. B., and Motter, A. E.
Be-yond word frequency: bursts, lulls, and scaling in the
temporaldistributions of words. PLoS ONE 4, e7678 (2009).
[32] Perc, M. Self-organization of progress across the century
ofphysics. Sci. Rep. 3, 1720 (2013).
Results Discussion Methods Controlled noise and discounting
free-riding Graph randomization Human annotation Alternative
metrics
Acknowledgments References