140 Characters in Search of a Meaning Incorporating Motivation into Altmetrics Mike Taylor Research Specialist .

140 Characters in Search of a MeaningIncorporating Motivation into Altmetrics

Mike TaylorResearch Specialist

http://orcid.org/0000-0002-8534-5985

The formulation and development of altmetrics has hitherto largely been rooted in computer science, with the early issues being the collection and presentation of social citations. However, there is a disconnect between the straightforward counting of mentions, and the interpretation of academic motivation and meaning.As altmetrics evolves into a sub-discipline, it is important for academics of all fields to engage with the debate, the emerging literature, and altmetrics’ adoption into the lingua franca of academic discourse and evaluation. Questions on the motivation and the value of public sharing require input from the social sciences: there will be cross-cultural and cross-discipline differences, gender issues and network bias. However altmetrics develops over the course of the next few years, these issues should be discussed in order to correctly position the debate within the scholarly community and to ensure benefits for all parties.

• Degree in psychology, mostly social• Most work is computer science

• An explanation of private social citation: to create a bookmark / reference on a networked application without awareness that this is public data

Observer bias

Changing nature of scholarly communications

1940s to present:• Publication• Discussion in institutions, conferences, labs• Citation• On-going private discussion => bibliometrics2008 to present:• Publication• Discussion in institutions, conferences, labs, social networks• Citation• On-going public and private discussion / activity =>

“altmetrics”

A brief background to altmetrics

• Four years old• Collection and analysis of bookmarks, social

media references, mass media references, recommendations, blog posts, videos that link to an article (mostly via a DOI)

• Sometimes includes page views and downloads (article level metrics)

Altmetrics…

…is a very broad term• Detection• Collection• Measurement• Analysis• Presentation• Interpretation

The scope of altmetric data

• Distinctly scholarly: Mendeley, Citeulike, Zotero

• Re-use: Figshare, Data Dryad• Technical: Github• Wider social: Twitter, Facebook, Delicious• Mass media: BBC, NY Times• Publisher usage data: ScienceDirect, Plos

What does it all mean?

• Meaning is not correlation• Altmetrics is not yet a big data problem (review of 5

years of papers show only 17% have accrued altmetric data, very few get 1000s of mentions)

• Some research using correlation, very little theory

• Validity, accuracy, applicability?• Not well suited for ‘simple’ search ranking

algorithm – ‘best paper’ concept would be absurd

A brief statement of the new problem

• We don’t know what people intend when they socially cite a paper

A brief statement of the old problem

• We don’t know what people intend when they socially cite a paper

…except…

Lessons from ancient history(1940s-present)

• There’s a lot of research into why people cite!• What do citation counts measure? A review of

studies on citing behavior (2008) Lutz Bornmann, Hans-Dieter Daniel

• Great review of motivating literature from 1960s onwards

• 150+ citations, none of them focused on altmetrics

• http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.4737&rep=rep1&type=pdf

Déjà vu all over again

• Literature, reviews, methodological approaches, data sets, theoretical frameworks

• Normative theory vs Social constructivist• “Scientists give credit to colleagues whose work

they use by citing that work” vs• “Scientific knowledge is socially constructed

through the manipulation of political and financial resources and the use of rhetorical devices”

Garfield’s early postulations (1962)

1. Paying homage to pioneers; 2. Giving credit for related work (homage to peers); 3. Identifying methodology, equipment, etc.; 4. Providing background reading; 5. Correcting one’s own work; 6. Correcting the work of others; 7. Criticizing previous work; 8. Substantiating claims; 9. Alerting to forthcoming work; 10. Providing leads to poorly disseminated, poorly indexed, or uncited work; 11. Authenticating data and classes of fact (physical constants, etc.); 12. Identifying original publications in which an idea or concept was discussed; 13. Identifying original publication or other work describing an eponymic concept or term (...); 14. Disclaiming work or ideas of others (negative claims); and 15. Disputing priority claims of others (negative homage)

Research over 40+ years…

• Good evidence for normative behaviour• Cronin (2005) “The weight of empirical

evidence seems to suggest that scientists typically cite the works of their peers in a normatively guided manner, and that these signs (citations) perform a mutually intelligible communicative function”

Lots of solid observations

• Affirmational citation - 10% to 90%• Assumptive type - 5% to 50%• Conceptual type - 1% to 50%• Contrastive type - 5% to 40%• Methodological type - 5% to 45%• Negational type - 1% to 15%• Perfunctory type - 10% to 50%• Persuasive type / ceremonial - 5% to 40%

Enormous variation in %

• Methodological (citation context analysis, author interview, author survey)

• Many disciplines• Classification variation

Formal citation vs social citation

• Formal citation is slow, and is part of academic culture (“show your reading…”)

• (Public) social citation is fast and derived from popular culture

• Formal citation is a good tool to understand scholarly impact

• Some altmetric data might be good for scholarly impact, some might be good for social impact

Characteristics of social citation

• Adding a paper to Mendeley (a reference manager with a ‘social’ network)

• Writing a blog• Posting a tweet

• Anonymous vs named, absent vs trivial vs substantial context, variation in author connection strength (etc), speed of responsiveness

Expanding social constructivist view

• Rhetoric is in real-time environment• Public citation is visible, findable, reference-

able• Explicitly builds a tangible network – “I’m your

follower, follow me”• Manipulative rhetoric?

Social constructivist and normativist

• Public social citation will be both (how much personal value would need to accrue to positively tweet something useless)

• Private social citation more like to be solely normative

• Requirement for a theory of social media?

The anatomy of a tweet

• Commentary / context• Author awareness• Network approval• Network construction• Passing on / retweets• Recommending• Informational

Context within an existing network

• But if the author is a colleague, a lover, a friend, a foe, a boss, an assistant…

• Social citation is intimate and social networks are curated

• Power, cultural, gender issues

Focus on the normative elements

• Reasonable implication that a cited paper has been read

• Social citation cannot have this implication:• “I have read”, “I intend to read”, “I might read”,

“I read other works by the authors”, “the abstract looks useful”, “I might skim read when looking for a citation”

• All are additional to other normative behaviour

Altmetric impact is not purely scholarly impact

• After 40+ years we can make some conclusions about formal citation

• After 4 years we can make no such conclusion about social citation

• Probably normative plus constructivist

Value-free classifications

• Social noise• Scholarly action• Scholarly commentary• Mass media• Re-use

• Looking for events, researching for a narrative

Altmetrics has many functions

1. Prediction of ultimate citation2. Measuring / recognizing component re-use /

preparatory work / reproducibility3. Hidden impact ( impact without citation )4. Real-time filtering / real-time evaluation5. Platform / publisher / institution comparison6. Measuring social reach / estimating social

impact

A matrix of data against functionSocial activity

Scholarly activity

Scholarly comment

Mass-media Re-use

Prediction x x

Re-use x

Hidden impact

x x x x x

Real-time filtering

x x

Comparison x x x

Social reach x x

Guesswork that needs verification and data!

The more data we can see

• The bigger the picture

scholarly activity (mean)

scholarly comment (mean)

log(social activity (mean))

mas media (mean)

Data from altmetric.com (c.100 articles since pubdate -2)

We really need the meaning to progress our understanding

• Altmetrics needs social scientists• Altmetrics is a bag of measurements of human

activity• Altmetrics cannot be solely left to computer

scientists• This is more than maths and code• Please: engage, research (willing research

partner: [email protected])