-
How do Learning Analytics “act” in Education? Simon Buckingham
Shum Knowledge Media Institute, Open University UK
simon.buckinghamshum.net twitter @sbskmi
ESRC Code Acts in Educa/on seminar
series h.p://codeactsineduca7on.wordpress.com
University of Edinburgh, 9th May
2014
h.ps://twi.er.com/Wiswijzer2/status/414055472451575808
-
Siri is smart…
“Find code acts in education”
-
Siri is smart…
“Find code acts in education”
-
h.ps://twi.er.com
/Wisw
ijzer2/status/414055472451575808 “Note: check
the
huge difference between knowing and
measuring…”
-
A word of introduction… What does this tell you?
-
Learning Technology
KMi, OU
AI & Argumenta7on
Learning Disposi7ons
Human-‐Centred Informa7cs
Learning Analy7cs
Seman7c Scien7fic Publishing
Dialogue / Issue / Argument
Mapping
A word of introduction… What does this tell you?
-
Critical scepticism to economic modelling
-
Critical scepticism to economic modelling
-
The conversation you don’t want to have with your learning
analytics wonk
LAK13 Panel: Educational Data Scientists: A Scarce Breed
http://people.kmi.open.ac.uk/sbs/2013/03/lak13-edu-data-scientists-scarce-breed
John Behrens (Pearson)
http://people.kmi.open.ac.uk/sbs/2013/03/lak13-edu-data-scientists-scarce-breed
-
Data ethics and societal readiness ny7.ms/1k5rY72
-
Data ethics and societal readiness ny7.ms/1k5rY72
-
In what senses do learning
analy1cs “act” in educa1on?
-
make the invisible visible
make solids permeable
make the ephemeral persistent
Educa7onal (indeed all)
Researchers
In what senses do learning
analy7cs “act” in educa7on?
-
make the invisible visible
make solids permeable
make the ephemeral persistent
Learning Analy7cs
In what senses do learning
analy7cs “act” in educa7on?
Educa/onal researchers have always done
this — what’s new is that
these tools are now in the
hands of those who actually
produce the ac/vity
-
outcome
In what senses do learning
analy7cs “act” in educa7on?
By changing the system dynamics
researchers / educators / instruc7onal
designers administrators / leaders /
policymakers
intent
-
outcome
In what senses do learning
analy7cs “act” in educa7on?
By changing the system dynamics
Faster feedback loops could enable
more rapid adapta7on: of agents’
behaviour, and of learning resources
and designs
researchers / educators / instruc7onal
designers administrators / leaders /
policymakers
intent
-
Delega7on of authority to define
goals, analy7cs, and implica7ons
In what senses do learning
analy7cs “act” in educa7on?
Distribution of power between educators, learners, leaders,
community…?
?
?
-
In what senses do learning
analy7cs “act” in educa7on?
What data, concepts and relationships do the analytics designers
seek to model?
Ontologically
-
Bowker, G. C. and Star, L. S. (1999). Sorting Things Out:
Classification and Its Consequences. MIT Press, Cambridge, MA, pp.
277, 278, 281
“Classification systems provide both a warrant and a tool for
forgetting [...] what to forget and how to forget it [...] The
argument comes down to asking not only what gets coded in but what
gets coded out of a given scheme.”
-
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning
Dispositions and Transferable Competencies: Pedagogy, Modelling and
Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics &
Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM. Eprint:
http://oro.open.ac.uk/32823
“A marker of the health of the learning analytics field will be
the quality of debate around what the technology renders visible
and leaves invisible.”
http://oro.open.ac.uk/32823
-
In what senses do learning
analy7cs “act” in educa7on?
What thresholds, samples, relationships, patterns, etc. do the
algorithms encode and seek? On what basis is a recommendation
engine proposing interventions?
Algorithmically
-
governingalgorithms.org Learning Analy7cs
In an increasingly algorithmic world
[…] What, then, do we talk
about when we talk about
“governing algorithms”?
-
governingalgorithms.org
A technology or an epistemology?
Barocas, S., Hood, S. and Ziewitz,
M. (2013). Governing Algorithms: A
Provoca1on Piece. Social Science
Research Network Paper 2245322. DOI:
h.p://dx.doi.org/10.2139/ssrn.2245322
Secrecy, obscurity, inscrutability
Agency, automa7on, accountabili7es
A typology of algorithms by genre?
The inscrutability of algorithms
Norma7vity, bias, values
-
In what senses do learning
analy7cs “act” in educa7on?
What meaning-making does the representation and interaction
design encourage?
Semio7cally
-
In what senses do learning
analy7cs “act” in educa7on?
epistemology
pedagogy assessment
Knight, S., Buckingham Shum, S. and Littleton, K. (In Press,
2014). Epistemology, Assessment, Pedagogy: Where Learning Meets
Analytics in the Middle Space. Journal of Learning Analytics. Open
Access Eprint: http://oro.open.ac.uk/39226
the ‘middle
space’ of learning analytics
in the context of computation...
What epistemological assumptions are shaping the assessment
regime, and hence the pedagogy? What questions are analytics used
to help answer?
http://oro.open.ac.uk/39226
-
In what senses do learning
analy7cs “act” in educa7on?
Analytics reports at the organisational and national levels come
with consequences at different scales — sometimes punitive, often
impacting millions of people.
Poli7cally
-
Figure from Doug Clow:
h.p://www.slideshare.net/dougclow/the-‐learning-‐analy7cs-‐cycle-‐closing-‐the-‐loop-‐effec7vely
(slide 5)
In what senses do learning
analy7cs “act” in educa7on?
All of the above are encapsulated
in any learning analy7cs deployment
-
28
What kinds of learners? What kinds
of learning?
What data could be generated
digitally from
the use context? How is it
‘cleaned’?
Does your theory predict pa.erns
signifying
learning?
What human +/or sooware interven7ons
/recommenda7ons?
How to render the analy7cs, for
whom, and will they
understand them?
What analy7cal tools could be used
to find such
pa.erns?
In what senses do learning
analy7cs “act” in educa7on?
-
Learning Analy7cs approaches act in
educa7on…
Ontologically Algorithmically Semio7cally
Systemically
Poli7cally
Authority?
-
>>> Learning Analy7cs cri7cal
template…
Ontology …
Algorithms …
Semiotics …
Epistemology …
Pedagogy …
Assessment regime …
System Dynamics …
Authority …
Politics …
-
examples
-
English primary school data RAISEonline
sta7s7cs
for a school, on ‘core’ subjects
(maths, literacy), a.endance and
demographics
-
Visualising “attainment” and “progress”
-
Which analytics could reflect the progress that ‘Joe’ has made
on so many other fronts other than his SATS?
34
-
student modelling by an adap1ve
tutor
-
Intelligent tutoring for skills mastery
http://oli.cmu.edu
http://oli.cmu.edu
-
37
https://grockit.com/research
Adaptive platforms generate fine-grained analytics on curriculum
mastery
https://grockit.com/research
-
analy1cs for learning disposi1ons
-
Qualities of effective learners Factor analysis of the
literature plus expert interviews: identified seven dimensions of
effective “learning power”, since validated empirically with
learners at many levels. (Deakin Crick, Broadfoot and Claxton,
2004)
-
Qualities of effective learners
40
-
ELLI: Effective Lifelong Learning Inventory
-
Quantifying learning dispositions agency; identity; motivation;
responsibility
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning
Dispositions and Transferable Competencies: Pedagogy, Modelling and
Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics &
Knowledge. (29 Apr-2 May, Vancouver). Eprint:
http://oro.open.ac.uk/32823
http://learningemergence.net/2012/04/30/learning-powered-learning-analytics
A wholis7c visual, intended to
build intrinsic mo7va7on, invi7ng
stretch, providing a new language,
provoking conversa7on that 7es to
the learner’s iden7ty
http://oro.open.ac.uk/32823http://learningemergence.net/2012/04/30/learning-powered-learning-analytics
-
Quantifying learning dispositions agency; identity; motivation;
responsibility
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
-
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsets
2020? personal data cloud generates my dispositional profile for
reflection >>> help me take responsibility for my own
learning
Shaofu Huang: Prototyping Learning Power Modelling in
SocialLearn
http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
Simon Knight:
http://people.kmi.open.ac.uk/knight/2014/02/knowledge-in-search
Social network patterns, teamwork effectiveness
and initiation of relationships
Questioning, arguing and search behaviours
reveal intrinsic curiosity and epistemic commitments
Tagging/sharing/blogging/social patterns
reveal how you see connections between
ideas
Perseverance, grit, tenacity; overcoming
panic/stress when challenged — through
behavioural and somatic traces
http://learningemergence.net/2014/03/01/assessing-learning-dispositions-academic-mindsetshttp://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposiumhttp://people.kmi.open.ac.uk/knight/2014/02/knowledge-in-search
-
Open source approach to analytics: towards true transparency and
emergence for complex systems?
http://www.solaresearch.org/mission/ola
http://www.solaresearch.org/mission/ola
-
conclusion analy1cs act in
educa1on — on mul1ple dimensions
an analy1cs approach perpetuates
an educa1onal worldview — so
let’s ensure this is
inten1onal – not accidental...