Learning Analytics - Improving Student Retention
Post on 27-Nov-2014
1896 Views
Preview:
DESCRIPTION
Transcript
SITS:Vision Annual Conference
Learning Analytics –Improving Student Retention
Paul Travill - Academic Registrar, University of Wolverhampton
Chris Ballard - Innovation Consultant, Tribal
SITS:Vision Annual Conference
“Learning analytics is the measurement, collection, analysis and reporting of data about learners and
their contexts, for the purpose of understanding and optimising learning and the environments in which it
occurs.”(George Siemens 2011)
What is Learning Analytics?
SITS:Vision Annual Conference
What is Learning Analytics?
Learning Analytics
Educational Data
Mining
Academic Analytics
Predictive modellingExtract value from big data sets
Business Intelligence applied to education at an institutional, regional and national level
Understand how students are learning and optimise the learning process
SITS:Vision Annual Conference
Contrasting Learning and Academic Analytics
Monitoring and benchmarking of university KPIs
SITS:Vision Annual Conference
Education and Big Data
SITS:Vision Annual Conference
Characteristics of Big Data
2011 Gartner Report
Variety VelocityVolume
SITS:Vision Annual Conference
Factors affecting retention
Preparation for higher
education
Academic Integration
Social Integration
Personal circumstancesEngagement
Demographic background
Course
SITS:Vision Annual Conference
Administrative Data Activity Data
Academic performance at
entrance
UCAS Application
Attendance
Engagement
Contact with support services
VLE Usage
Library Usage
Proximity Door access
Social background Module Grades
Course Enrolment
Fees
Engagement and Academic Integration
Predictive Model
Demographics Contact with tutors
Campus PC Usage
Social interaction
Possible future data source
Student factors
SITS:Vision Annual Conference
Administrative Data• Student
Administration System
• Known at time of enrolment
Activity Data• User interaction
with a system• Patterns of usage• Real time• Collected at scale• Change over time
Initial assessment of
risk
On going assessment of
risk
SITS:Vision Annual Conference
The dataset
• Data Warehouse• Data set spanning 2010/11 – 2011/12
academic years• Imbalanced data
SITS:Vision Annual Conference
Staff
Admin Data
Activity DataPredict likelihood of
withdrawal
Predict module grades
View profile of student interactions
Module Outcome Model
Retention Model
Activity Profile
Stud
ent
Comparison to similar students
Cluster students
Which things can we change that could make a difference?
SITS:Vision Annual Conference
SITS:Vision Annual Conference
Module: Basic Broomstick SkillsHarry is likely to achieve a grade D. Issues:VLE use is very low compared with the better performing students,Distance from home suggests he should have a higher VLE use profileSupport Recommendation:Suggest attendance at the additional Broomstick Study Skills sessions (Wednesday at 12.00 in the library) Click to make booking.
Module: Quidditch magic spellsHarry is likely to achieve a grade B. This matches the profile of other students in the profile cluster. Remember to encourage him to keep up the work!
Harry PotterCourse: BWiz QuidditchNOTES FOR PERSONAL TUTOR
SITS:Vision Annual Conference
ACTIVITY DATA
SITS:Vision Annual Conference
Activity DataStudent ID Date Time Asset ID
0000001 01/09/2012 12:03:01 1
0000001 01/09/2012 12:05:06 34
0000001 05/09/2012 16:46:23 17
0000005 17/10/2012 19:56:01 73
… … … …
Student ID Date Module Number of Transactions
0000001 01/09/2012 Abc 2
0000001 02/09/2012 Abc 0
0000001 03/09/2012 Abc 0
0000001 04/09/2012 Abc 0
0000001 05/09/2012 Abc 1
0000005 17/09/2012 Bcd 7
… … … …
Transactions Time Series
SITS:Vision Annual Conference
Activity Data Goals
• Convert to Time Series• Pre-process time series (e.g.
smoothing)• Derive measures which describe the
“shape” of the interactions• Use measures to help understand
whether some patterns of interaction are indicative of poor engagement
SITS:Vision Annual Conference
Extracting meaningful information from Activity Data
• Need to distinguish between students who are regular users of the service, and those who have sporadic high volumes of access (but aggregate volume may be similar
• Acts as a proxy to how well the student is “engaged” with the service
is better than
But may have similar overall numbers of transactions
SITS:Vision Annual Conference
Symbolic Aggregate approXimation (SAX)
abcdbbbdeaaddbae
Encodes the shape of the time series as a series of character strings
Enables us to cluster together students with similar interaction patterns, or classify interactions (as indicative of students who ultimately withdrew)
Turns out to work well for high volumes of interactions, but not so well for intermittent time series as there is less “shape” to encode.
Smoothing the time series may help
SITS:Vision Annual Conference
Derive high level measures
• Proportion of days accessed resource• Average number of transactions per
day accessed• Run Length Distance Ratio
Turns out to work better for Library and VLE activity datawhere interactions are much more intermittent
SITS:Vision Annual Conference
School resource profiles• Low VLE usage does not mean the same thing for every
student!• Need to weight Library and VLE features to take into
account different resource profiles
SITS:Vision Annual Conference
Challenges with activity data
SITS:Vision Annual Conference
BUILDING THE MODEL
SITS:Vision Annual Conference
SITS VLE Library Active Dir.
Demographics, Enrolment, Activity Data, …
Test examples (30%)
TransformPredictions
New
dat
a
1. Train model
2. Test model
Model
Data Warehouse
Calculated FeaturesDerived Features
Training examples (70%)
Other Data SetsOther Data Sets
SITS:Vision Annual Conference
Performance of the model
Admission
• Demographics
• Course• Academic
performance on entry
• Distance
Semester 1
• Modules Failed
• Modules Passed
• Credit points
VLE ActivityLibrary Activity
VLE ActivityLibrary Activity
Lower Performance
Higher Performance
SITS:Vision Annual Conference
Challenges with the model
Imbala
nced classe
s
Imbalanced classes require the need to adjust data to ensure model has adequate performance
Training d
ata
Its all about the training examples and features!
History
Training the model with examples representing the progress the student makes at different times
Perfor
mance
Performance improves as the academic year progresses
SITS:Vision Annual Conference
Chris BallardInnovation Consultant, Tribal
twitter: @chrisaballardblog: triballabs.net
top related