EASICONNECT @ CQUni Damien Clark Twitter: @damoclarky http:// damos.world Colin Beer Twitter: @beerc http:// beerc.wordpress.com Rolley Tickner Twitter: @rolley http:// rolleys.wordpress.com
EASICONNECT @ CQUni
Damien ClarkTwitter: @damoclarkyhttp://damos.world
Colin BeerTwitter: @beerchttp://beerc.wordpress.com
Rolley TicknerTwitter: @rolleyhttp://rolleys.wordpress.com
Is a Team Effort
Colin Beer Rolley
Tickner Me!
Introductions
Is a Team Effort – Team Background
Teaching
Research
Programming Data
AnalysisSystem Administration
Graphic Design
DB Design
Complementary blend of skills:
Self Appointed:Ed Tech Team (not in org. chart)
Central L&T
The teachers view of a disengaged student
Looking around your classroom - what are the signs?
http://farm1.static.flickr.com/32/64368770_7bce91daf6_b.jpg
The teachers view of a disengaged fully online student
https://www.flickr.com/photos/wfryer/6727304641/
The Institution
CQU Context (Brief Overview)
Where is CQU?
Our footprint…
Where is CQU?
Where is CQU?
My team located here ->
Where are our HE students?
20+ Locations
Source: CQU Student Records
+ VET - 2014
Distance Ed
International
Source: CQU Student Records
Distance Ed
International
Where are our HE students?
DemographicsP
redominantly
Regional/R
emote
Enrolm
ents
DemographicsP
redominantly Low
/Mid S
ES
CQU – Central Queensland University
http://www.beingabusinesscelebrity.com/when-business-gets-overcomplicated/
CQU – Complicated Queensland University
Complementary Tools
Early Alerts Student Indicators (EASI)
Demonstration
Estimate of Success (EOS)
Emphasises communication between teachers and students rather than accurate prediction
https://xkcd.com/612/
Aggressively weighted towards risk of failure More false positives than false negatives
http://www.hangthebankers.com/5-top-economic-advisers-forecast-world-war/
http://sorenandersson.com/a-sustainable-nudge/
Nudge
Nudge (verb):1. to push or poke (someone) gently, esp with the elbow, to
get attention; jog2. To push slowly or lightly3. To give (someone) a gentle reminder or encouragementhttp://www.thefreedictionary.com/nudge
Nudging in Business
Amazon Marketing
http://sorenandersson.com/a-sustainable-nudge/
Nudging – what’s the problem with this nudge?
{$student.firstname},
You have failed to connect regarding <coursename>. Recent phone calls and emails to you have gone unanswered. YOU ARE AT RISK OF FAILING THE COURSE!
{$staff.firstname} {$staff.lastname}
Nudging – what’s the problem with this nudge?
{$student.firstname},
You have failed to connect regarding <coursename>. Recent phone calls and emails to you have gone unanswered. YOU ARE AT RISK OF FAILING THE COURSE!
{$staff.firstname} {$staff.lastname}
http://discoversociety.org/2015/12/01/policy-briefing-the-nudge-business/
Not encouraging
Pestering?
http://sorenandersson.com/a-sustainable-nudge/
Nudging – what’s the problem with this nudge?
Dear {$student.firstname},
This is the third time I have emailed you regarding your involvement with <coursename> . It would appear that you have not as yet accessed the course Moodle website despite my messages. If you have no intention of successful completion of this course, I would suggest that you drop it immediately. Census day is on Tuesday, and if you do not drop it before then, you will be charged tutition fees for the course, and ultimately be awarded the grade of FAIL if your current lack of activity continues.
If you choose to continue in the course, would you please contact your lecturer and arrange to design a study plan that will ensure that you quickly catch up with the course. If you are unwilling to do this, I strongly urge you to drop the course.
Regards, {$staff.firstname} {$staff.lastname} Course Coordinator: {$course.code} {$staff.email}
Nudging – what’s the problem with this nudge?
“This is the third time I have emailed you regarding your involvement with <coursename> . It would appear that you have not as yet accessed the course Moodle website despite my messages. If you have no intention of successful completion of this course, I would suggest that you drop it immediately… “
http://www.flickr.com/photos/cuppini/2757884114/
Teacher frustration is showing.
Nudging – what’s the problem with this nudge?
Hi, We are sending this little nudge as our records show that you may be needing some encouragement to engage more in our course ‘<coursename>'. Research has shown that students who engage in the moodle site at least 3 times a week are less likely to fail or have issues with falling behind. Please if you are having any problems contact us. We do understand that it is week 1 and there are many things to get our heads around including the technology and the set up of the Moodle site. So please take heed of our nudge and seek help if you need it or rearrange your schedule so you can actively take part in this interesting and worthwhile course as part of your learning journey. regards <teacher names>
http://sorenandersson.com/a-sustainable-nudge/
Nudging – what’s the problem with this nudge?
Hi, We are sending this little nudge as our records show that you may be needing some encouragement to engage more in our course ‘<coursename>’…
http://www.flickr.com/photos/cuppini/2757884114/
Not personalised
Examples of Use (EASI)
http://www.doityourself.com/scat/handtools/
Following slides include more effective examples of real nudges
Examples of Use (EASI) – Case Management
http://www.doityourself.com/scat/handtools/
Case management between teachers
Examples of Use (EASI)
http://www.doityourself.com/scat/handtools/
Assessment Non-submission
http://www.doityourself.com/scat/handtools/
Shared notes
Examples of Use (EASI) – Shared Notes
Examples of Use (EASI) – Shared Notes
http://www.doityourself.com/scat/handtools/
Shared notes
The teachers view of a disengaged fully online student
Remember this?
http://www.doityourself.com/scat/handtools/
Examples of Use (EASI)
“We can see at a glance the level of student activity on Moodle - this is one of the most valuable aspects of Easi Connect. We can see exactly what weeks students have been active, and when their activity might have dropped off.”
http://www.doityourself.com/scat/handtools/
Examples of Use (EASI) – Student Responses
http://www.doityourself.com/scat/handtools/
Examples of Use (EASI) – Student Responses
http://www.doityourself.com/scat/handtools/
Examples of Use (EASI) – Student Responses
http://www.doityourself.com/scat/handtools/
Examples of Use (EASI) – Student Responses
Why EASI? Motivation?
CQU Ouch!
!
Source: http://highereducationstatistics.education.gov.au/
2003 - 2009
Database access
LMS Support
Unintended consequences
We started noticing trends in the data
http://farm3.static.flickr.com/2672/3684420623_796346b171.jpg
http://www.flickr.com/photos/dragnfly78/235652252/
2006 2007 2008 2009 2010 2011 2012 2013 20140
500
1000
1500
2000
2500
3000
16 15 22 1667
271
773
1460
2430
Google Scholar publications using the term "learning analytics"
http://flickr.com/photos/boskizzi/3241710/
http://flickr.com/photos/boskizzi/3241710/
Oz/NZ Horizon Report
Year Time Frame Label
2010 4 to 5 years Visual data analysis
2012 1 year or less (#2)
Learning analytics
2013 1 year or less (#1)
Learning analytics
2014 2 to 3 years Learning analytics
http://www.flickr.com/photos/mikecogh/5959192031/
I’m not familiar with (m)any universities thathave taken a systems-level view of LA.
http://bit.ly/16uz8vU
Most of what I’ve encountered to date is specificresearch projects or small deployments of LA. I haveyet to see a systemic approach to analytics use/adoption
http://www.flickr.com/photos/johnhaydon/5042881685/
Basing decisions on data and evidenceseems stunningly obvious(Siemens and Long, 2011, p.
31)
CQUni’s Moodle site doesn’t have MAV
No MAV links
MAV installed in browser
MAV Staff Self-install Page
Firefox Add-ons / User Scripts
MAV installed in browser
Moodle Activity Viewer – Under the HoodNo changes toMoodle code ✔
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Responses:
“I can see which links are being clicked on and which students are clicking on those links and I can see whether or not I am wasting my time producing a video that no-one ever watches or if there is something that has got the students’ attention and there is a lot of people clicking on it, I can see what that is.”
http://www.doityourself.com/scat/handtools/
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Examples of Use (MAV)
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Responses:“The additional feature - the MORE icon - which allows us to access this information from other parts of Moodle, such as individual assignments, is also proving really valuable. ”
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Examples of Use (MAV)
“We can access the Handbook from EASI CONNECT - this provides us with useful contact information e.g. phone numbers, as well as a view of their study history.”
http://www.doityourself.com/scat/handtools/
Responses:
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
Examples of Use (MAV)
http://www.doityourself.com/scat/handtools/
IT Management – Making Changes
Recommendation:“Implement Vanilla systems (remove or reduce customisations unless essential or demonstrated benefit over cost)” (CQUniversity Information Technology Department Restructure, 2013)
http://www.praqma.com/services/plainvanilla
https://twitter.com/vanillagames
Moodle Activity Viewer – Under the HoodNo changes toMoodle code ✔
Remember this?
IT Management – Making Changes
http://elevator.empowernetwork.com/blog/lifes-too-short-to-be-plain-vanilla
We think…
Practice Underpinned by Research
Breaking BAD to bridge the reality/rhetoric chasm. (ASCILITE2014)
Three paths for learning analytics and beyond: moving from rhetoric to reality (ASCILITE2014)
The IRAC framework: Locating the performance zone for learning analytics. (ASCILITE2013)
David Jones, Colin Beer, Damien Clark & Rolley Tickner
Practice Underpinned by Research
Indicators of engagement (ASCILITE2010)
Academic involvement in the LMS (ASCILITE2010)
Analytics and complexity: Learning and leading for the future (ASCILITE2012)
Practice Underpinned by Research
Choose the appropriate path: Do it With the AcademicEffective Learning Analytics uses a PIRAC
If you are SET in your ways, sometimes you should break BAD
Purpose Informatio
n
Representation Affordance
s
Change
Bricolage
Ability Change
Distribution
Decision-making Philosophy
http://elevator.empowernetwork.com/blog/lifes-too-short-to-be-plain-vanilla
Have been asked:
“Can I have EASI send out automatic emails to my students?”
Our response: “No.”
I’ll explain why with an analogy
Aviation Industry – 2 major aircraft manufacturers
https://commons.wikimedia.org/wiki/File:Airbus_A380_inbound_ILA_2006.jpg
Airbus
Aviation Industry – 2 major aircraft manufacturers
https://commons.wikimedia.org/wiki/File:Boeing_787_Dreamliner_N787BX.jpg
Boeing
Automation Philosophies
http://www.aerospaceamerica.org/Documents/Aerospace America PDFs 2015/July-August 2015/AA_Jul-Aug2015_Feature3_BoeingVsAirbus.pdf
Assistant Professor Clint R. Balog
“Airbus’ philosophy is automation-centric”
“Boeing’s is human-centric”
“This can be seen in Airbus’ publicly available philosophy statement, which says that ‘all aircraft have physical limits that they must not exceed’ and that such flight-envelope limits ‘are not to be exceeded during normal operations.’”
“Automation, according to Boeing, is ‘to assist, but not replace, the flight crew member responsible for safe operation of the airplane.’”
Automation Philosophies
http://www.aerospaceamerica.org/Documents/Aerospace America PDFs 2015/July-August 2015/AA_Jul-Aug2015_Feature3_BoeingVsAirbus.pdf
Assistant Professor Clint R. Balog
“Airbus’s philosophy appears to be that automation generally knows better than the pilot and should, under most circumstances, have the final decision authority. Boeing takes the view that the pilot is best able to evaluate individual, and possibly unique, operational circumstances and should make the ultimate decision.”
https://commons.wikimedia.org/wiki/File:Boeing_787_Dreamliner_N787BX.jpg
We aspire to Boeing’s philosophies of decision-making using learning analytics
Decision-making Philosophy
Our team Boeing takes the view that the teacher pilot is best able to evaluate individual, and possibly unique, teaching operational circumstances and should make the ultimate decision.
Lessons & Outcomes
Usage of the technology
Ethical Implications
https://www.flickr.com/photos/icma/3635981474
Increased Student Activity
Usage of EASI & MAV
https://en.wikipedia.org/wiki/Choice
Staff @ CQU choose to use EASI & MAV
Usage of EASI & MAV
https://en.wikipedia.org/wiki/Choice
EASI Usage
MAV Usage
281k nudges20k individual students2k course offerings
333k page views
Since Term 1, 2014
280 staff
2k course offerings
898 courses
Study Activity Resulting from Nudges
Control: Students who didn’t receive nudges: Mean +4 clicks at same weeks
Students who received nudges:
Mean +27 clicks 3 weeks after comparedto 3 weeks prior to nudge
p < 0.0001
Two tailed independent samples t test
df = 132164
t = 11.8
http://www.stanwicklakes.org.uk/northampton/honey-bees-buzzing-after-louise-mensch-visit/
Learning Analytics - Ethical considerations
Data ownership
SurveillanceLabeling
3 Findings from unpublished paper(under peer review)
http://quotesgram.com/funny-ethics-quotes/
Our advice to others
http://www.itsnotbadatall.com/funny_pictures/html/Funny_Advice.html
The predictive potential of learning analytics is overrated. Beware of the hype!
Our advice to others
The social/technical system that encapsulates learning analytics is not complicated, it’s complex.
http://www.dumpaday.com/funny-pictures/random-funny-pictures-36-pics-5/attachment/funny-parent-advice-twilight-2/
The predictive potential of learning analytics is overrated. Beware of the hype!
Our advice to others
The socio-technical system that encapsulates learning analytics is not complicated, it’s complex.
http://quotesgram.com/funny-advice-quotes/
The predictive potential of learning analytics is overrated. Beware of the hype!
Less of a focus on plan-based approaches to adoption and more on real-time situation awareness and sense making
Our advice to others
The socio-technical system that encapsulates learning analytics is not complicated, it’s complex.
http://quotesgram.com/funny-advice-quotes/
The predictive potential of learning analytics is overrated. Beware of the hype!
Less of a focus on plan-based approaches to adoption and more on real-time situation awareness and sense making
Balance PIRAC
Incidentally…
CQU Yay!!
Causality?
Source: http://highereducationstatistics.education.gov.au/
Thanks!
https://coub.com/view/4pyf0