Learning Analytics: A General Introduction and Perspectives from the UK

Dr Doug ClowInstitute of Educational Technology, The Open University, [email protected]@open.ac.uk

Læringsanalyse: en allmän introduktion

och perspektiv från Storbritannien

Learning Analytics: A General Introduction

& Perspectives from the UK

Dr Doug ClowInstitute of Educational Technology, The Open University, [email protected]@open.ac.uk

33

You are free to:copy, share, adapt, or re-mix;

photograph, film, or broadcast;

blog, live-blog, or post video of

this presentation provided that:You attribute the work to its author and respect the rights and licences associated with its components.

4

Using Lanyrd

The Lanyrd social directory of events provides access to information about:

• Events• Speaker• Participants

The entry for this event may include access to:

• Slides• Twitter archives• Reports

Feel free to add your details

http://bit.ly/1v0XT05

1. learning analytics2. at the Open University3. in UK schools

learning analytics

Photo (CC)-BY-NC-SA tim_d https://www.flickr.com/photos/tim_d/1840189287

“The most important single factor influencing learning is what the learner already knows. Ascertain this and teach [them] accordingly.”

– David Ausubel, 1968

What is learning analytics?

• the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs – First International Conference on Learning Analytics And Knowledge (LAK11), Banff, Alberta, Feb 27-

Mar 1, 2011

Photo (CC)-BY Cris: http://flickr.com/photos/chrismatos/6917786197/

Photo public domain: http://commons.wikimedia.org/wiki/File:DESYNebelkammer.jpg

- Erik Duval http://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/

“collecting traces that learners leave behind and using those traces to improve learning”

“feeding back the data exhaust”

Big Data in Education

Photo (CC)-BY Iain Watson http://www.flickr.com/photos/dagoaty/3329699788/

Clow, LAK12, 2012

(cc) Doug Clow http://dougclow.org

• Predictive modeling– Datamining, Blackboard

• Place students in one of three risk groups– traffic light / signal / robot

• Trigger for intervention emails• Dramatic retention improvements• Consistent grade performance improvement

14

control

Photo (CC)-BY Andy Roberts https://www.flickr.com/photos/aroberts/3035796

surveillance

15

support

Photo (CC)-BY-NC-SA Drew Bennett https://www.flickr.com/photos/abennett96/2710211041

guidance

16

M4 Motorway Cameras

Diagram (CC0) http://en.wikipedia.org/wiki/File:British_Isles_Euler_diagram_15.svg

17

M4 Motorway Cameras

Photo (CC)-BY Phillip Williams http://www.geograph.org.uk/photo/1342357

18

M4 Motorway Cameras

19Image (cc) Darwin Bell http://www.flickr.com/photos/darwinbell/296553221/

“The predictive model was used as a trigger for intervention emails to the student.”

20Image (cc) Darwin Bell http://www.flickr.com/photos/darwinbell/296553221/

From: [email protected] are in trouble. The computer predictive model gives you a 87.4322% chance of failing this course. You must see a tutor immediately.

21Image (cc) Darwin Bell http://www.flickr.com/photos/darwinbell/296553221/

From: [email protected] are in trouble. The computer predictive model gives you a 87.4322% chance of failing this course. You must see a tutor immediately.

Hi AlexAre you Ok? I noticed you haven’t logged on this week, and I know you struggled with the last assessment. We can work through this together - let’s have a chat as soon as possible.Pat.

Glasswinged butterfly, ? Greta oro Photo (CC)-BY-NC-ND by Greg Foster on Flickr http://www.flickr.com/photos/gregfoster/3365801458/

Principles

• Privacy• Data protection• Ethics• Transparency

• LAK conferences• LASI workshops• Flare local meetings• Storm PhD training• Journal of LA• … and more!

www.solaresearch.org

24

InternationalEducational Data MiningSociety

• Annual conference IEDMS• Journal of EDM• www.educationaldatamining.org

www.laceproject.euLearning Analytics Community Exchange (FP7)

• Coordination and Support• Evidence Hub• Events: SoLAR Flare, 24 Oct 14, UK• Publications, briefings, webinars

at the Open University

The Open University

• Largest university in the UK• 200,000 students• > 5,000 tutors• > 1,000 academic staff• Supported open learning• OpenLearn, YouTube,

iTunesU, FutureLearn

Milton Keynes

At scale, each year

~400 courses200,000 students

> 1 million assignments> 1 billion views of OU/BBC coproductions

> 3 million Moodle transactions per day

Photo (cc) Marieke IJsendoorn-Kuijpers http://www.flickr.com/photos/mape_s/333862026//

http://www.jumpoffthescreen.com/analytics.php

Analytics ProjectIntervention and Evaluation

Data VisualisationsEthics FrameworkPredictive ModellingLearning Experience Data

Professional DevelopmentSmall Data Student Tools

Photo (cc) jeroen bennink http://www.flickr.com/photos/jeroenbennink/2355768494/

To reduce dropout:• What will we do?• What change should we see?• Can we see that before the end of the course

• e.g. VLE data, assessment data• Can we test whether it works?

Intervention & Evaluation Framework

• Data Protection• Privacy• Transparency (related to Subject

Access requests)• Whether students should be able

to opt in/out• De-identification of data• Timeliness and Duty of Care

(keeping data up to date)• Access to data (who should have

access to the data, etc.)• Students abusing the system by

misinformation

Ethics

• The use of student data outside OU systems (Facebook, Twitter, etc.)

• Analysis of the data and the methods used (what assumptions are used to create the algorithm for the predictive model, should there be an independent audit?)

• Purpose of applying a learning analytics approach

• Profiling of students• How will it be done?• What do we tell students?• Should we tell students? – Students

may feel ‘at-risk’/labelled

Glasswinged butterfly, ? Greta oro cc licensed ( BY NC ND ) flickr photo by Greg Foster: http://www.flickr.com/photos/gregfoster/3365801458/

cc licensed ( BY) flickr photo by Karen Roe: http://www.flickr.com/photos/karen_roe/4916422687/

Predictive modelling

• To alert tutors• Financial planning• Quality assurance

– Was retention better than predicted?

• Select students–Demographics, module data,

VLE data (inc. key activity)• Trigger interventions

–Emails, notes to tutor, etc

Student Support Tool

Data Interpreter in each Faculty

cc licensed ( BY) flickr photo by Randy Robertson: http://www.flickr.com/photos/randysonofrobert/337922766/

36Photo (CC)-BY-NC-SA Moon Lee on Flickr https://www.flickr.com/photos/imagezen/65199660

Self-service data reports

Data Wranglers

human sense-makers

Photo CC (BY-NC) Alan English CPA: https://www.flickr.com/photos/alanenglish/4198114139

users data

users data

users data

futurelearn.com

in UK schools

Context

• National Curriculum• National testing (SATs, ages 7, 11, 14)• League tables• Analytics for tracking and monitoring

43Photo (CC)-BY Thomas Galvez on Flickr https://www.flickr.com/photos/togawanderings/14212266277

School dashboards (Google Images)

• Maybe chop the first slide about this.

44

45

final words

47

48

Dispositions analytics

• Learning dispositions– Resilence

• ‘Learning power’: Effective Lifelong Learning Inventory

• “A framework for a coaching conversation which moves between identity and performance”

• Schools to Graduate schools

Buckingham Shum and Deakin Crick, 2012 (LAK12)

49

ELLI Teacher view

Buckingham Shum and Deakin Crick, 2012 (LAK12)

50

Enquiry Blogger

51

Enquiry Blogger teacher dashboard

finally

catnip for senior

managers

Photo (CC)-BY Dylan Ashe https://www.flickr.com/photos/ackook/3929957511/

54

“When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science”

– William Thomson, Lord Kelvin

55

Not everything that can be counted counts.Not everything that counts can be counted.

– William Bruce Cameron

Photo (CC)-BY Paul Stainthorp https://www.flickr.com/photos/pstainthorp/5497004025

Not everything that can be counted counts.Not everything that counts can be counted.

– William Bruce Cameron

Photo (CC)-BY kaybee07 on Flickr https://www.flickr.com/photos/kurtbudiarto/7026555821

Thanks to:

People:• OU Learning Analytics: IET Student Statistics and Survey Team, Gill Kirkup and

the other Data Wranglers, Kevin Mayles, Belinda Tynan, Simon Buckingham Shum, Rebecca Ferguson, Bart Rientes, Sharon Slade, Kelly Bevis, many others

• LACE: Rebecca Ferguson, Simon Cross, Michelle Bailey, Rebecca Wilson, Evaghn De Souza, Natalie Eggleston, Oliver Millard, Gary Elliot-Citigottis, and our project partners.

• The learning analytics community, including SoLAR, IEDMS, those I’ve met at LAK and LASI

Funders:• LACE: European Commission 619424-FP7-ICT-2013-11

58

What Did You Learn Today?

What did you learn today?Feel free to share with others (on Twitter if you have access – use #laceproject)

59

“Learning Analytics: A General Introduction and Perspectives from the UK” by Doug Clow, Institute of Educational Technology, The Open University, was presented at Skolverket, Stockholm on 9 October 2014.

@[email protected]

This work was undertaken as part of the LACE Project, supported by the European Commission Seventh Framework Programme, grant 619424.

These slides are provided under the Creative Commons Attribution Licence: http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.

www.laceproject.eu@laceproject

cc licensed ( BY ) flickr photo by David Goehring: http://flickr.com/photos/carbonnyc/33413040/

61

Data Wrangling

users data

users data

users data

62

What data do we have about learners?

• Demographics– Gender, age, ethnicity, socio-economic status, address

• Previous educational experience– Schools, grades, results

• Grades, scores, achievements, struggles• Attendance, location

– Smart cards, proximity detectors

• Online tracking– VLE / LMS data: views, posts, interactions, quiz results

• Other online activity– Cross-tracking cookies

• … more every week.

63

What can we do with that data?

• Identify students who need help– Simple or predictive

• Trigger interventions– Via tutor, or direct

• See which interventions work• Suggest resources or source of help

– Learners like you found this helpful– This person might be able to help you

64

Not everything that can be counted counts.Not everything that counts can be counted.

– William Bruce Cameron

Photo (C) The Office of His Holiness the Dalai Lama