Current State and Future Trends:A citation network analysis of the Learning Analytics Field

S. DawsonD. Gasevic

G. SiemensS. Joksimovic

Current State and Future Trends:A citation network analysis of the

Learning Analytics Field

Goal

• Citation analysis and structural mapping to gain insight into the influence and impact within LA – a snapshot of LA through analysis of articles and

citations (LAK conferences and special issues)

Context

• Although much potential and excitement:– to date LA has served to identify a condition, but

has not advanced to deal with the learning challenges in a more nuanced and integrated manner

Aim

• Identify emergence of influential trends and hierarchies in the field

• Commencement point (Leah):– a foundation for future work– identify promising areas of research– Identify under represented disciplines– Improve integration across disciplines and theory

and practice

Context• Learning analytics:

– has emerged as a field (maturation)– multi-disciplinary– often mis-represented and poorly understood

• (Academic analytics; business intelligence; assessment analytics; social analytics; web analytics; educational data mining)

Approach

• Bibliometrics measure the impact/influence of an author or article using various citation analyses

• Garfield 1955 (Impact)

Ding, Y. (2011). Scientific collaboration and endorsement: Network analysis of coauthorship and citation networks, Journal of Informetrics, 5(1), 187-203

Data

• LAK11, 12, 13• Special issues – ETS, JALN, ABS• Data analysis

– Citation counts– Author/citation network analysis– Contribution type– Research methods– Author disciplinary background

Approach

• Citation and author networks:– Identify the prominent research– Identify linkages between disciplines and authors– Identify diversity of research genres

Citation analysis

• The use of citations long been used to measure impact– Core output of research is publications– As research grows, output (papers) further build

on other associated works (citations)– “quality” as quantity of citations

• Identify areas of prominent research activity

Citation analysis

• Criticisms– Cronyism– Self citations– “Rich get richer”

By the way some great refs

Gašević, D., Zouaq, A., & Janzen, R. (2013). “Choose Your Classmates, Your GPA Is at Stake!” The Association of Cross-Class Social Ties and Academic Performance. American Behavioral Scientist 57 (10), 1460-1479

Siemens, G. (2013). Learning Analytics The Emergence of a Discipline. American Behavioral Scientist 57 (10), 1380-1400

Dawson, S., Tan, J., & McWilliam, E. (2011). Measuring creative potential: Using social network analysis to monitor a learners' creative capacity. Australasian Journal of Educational Technology 27 (6), 924-942

Citation analysis

Citation analysis

• Ok Some faults – broadly accepted that quality is linked to the number of citations

Citation analysis

• Number of citations (Highly cited articles)– Predominately conceptual and opinion papers

(e.g. Educause)– Methods (Wasserman and Faust)– Few empirical studies

• Citation and author networks:– Illustrate linkages and disciplines

Example author networks

Paper 1: Liz, Julie, Sara

Sara

JulieLiz

Example author networks

Paper 1: Liz, Julie, Sara

Sara

JulieLiz

2 Degrees

Example citation network

Paper 1: Liz, Julie, SaraCites: Liam & Luka

Sara

JulieLiz

Liam

Luka

Citation network

LAK conferenceJournals

Citation networks

• Citation network moderate level of clustering– Consistent across LAK proceedings– Few strong connections? – Degrees low – indication of diverse and

inconsistent literature sources– Degrees (increasing) from LAK11 to 13

Author networks (LAK)

Author networks (Journal)

Author networks

• Author networks – small cliques with few highly connected nodes

• For an interdisciplinary field still largely disciplinary clustered

Paper classification

• Schema from Info Systems (6 categories)

1. Evaluation research – (e.g. case study empirical)

2. Validation research – (e.g. testing theory/ method/ solution empirical)

Glass, R.L., et.al, 2002. Research in software engineering: an analysis of the literature. Information and Software Technology 44, 8, 491-506

Paper classification

3. Solution proposal (solution/ technique to address an issue)

4. Conceptual proposal (e.g. frameworks)

5. Opinion(well its my opinion/argument)

6. Experience (Let me tell you a story)

7. Panel/workshop

Author Background

Extensive search – if home dept not listed in author details

Paper classification (LAK)

• Dominated by evaluations of research (journals)

• Proposal of solution dominates conference

Paper classification (Journal)

• Dominated by evaluations of research (journals)

• Proposal of solution dominates conference

Paper classification

• Dominated by computer science (LAK)• Greater number of education researchers in

journals– Reflection of special issues– Reflection of priority sites for publications

• Largely conceptual and opinion publications

Methods classification

• Qualitative• Quantitative• Mixed methods• Other

Methods classification (LAK)

• Other dominates – reflects high number of opinion and conceptual papers

Methods classification (Journals)

Methods classification

• Other dominates – reflects high number of opinion, experience and conceptual papers

Conclusions

• The field is in its infancy– Citations still predominately opinion and

definitional pieces– Clustering and degrees– Few number of empirical studies cited but this is

growing• Mature fields greater examples of validation

research and importantly critiques of studies

Conclusions

• Computer scientists dominate LAK proceedings– Need to look at how other voices are heard

• Education research dominates Journals– Reflection of broader priorities?

Conclusions

• Early work need to extend• Structural mapping and citation analyses more

common and more sophisticated.• Raise awareness

– Inform practice– Build connections– Foster further empirical work

Conclusions

• Understanding our field we can better advance our field.

• Question: To what extent can we use these analyses to architect the development of the field?

Questions

• Next steps:–Broader scope (extend network

analyses)–Keyword clustering–Citation location–Incorporate multiple citations/ paper

Questions

• To what extent can we use these analyses to architect the development of the field?

• [email protected]• [email protected]• [email protected]• [email protected]