The Use of ICT for the Assessment of Key Competences Irish Presidency Conference Better Assessment and Evaluation to Improve Teaching and Learning 19-20 March 2013 Christine Redecker Yves Punie
Apr 01, 2015
The Use of ICT for the Assessment of Key Competences
Irish Presidency Conference
Better Assessment and Evaluation to Improve Teaching and Learning
19-20 March 2013
Christine RedeckerYves Punie
IPTS: Part of Joint Research Centre of the EC: 7 Research Institutes across Europe
Mission: “to provide customer-driven support to the EU policy-making process by developing science-based responses to policy challenges that have both a socio-economic as well as a scientific/technological dimension”
IPTS- Who we are
Future Learning Spaces in 2020
2006 2007 2008 2009 2010 2011 2012 2013
Learning 2.0 in formal E&T Learning 2.0 in informal settings
Creativity & Innovation in schools in EU
eLearning in the EU10 MS
- Digital Competence- Active Ageing & Learning
Tell-Net Teacher Networking
Review ICT
Impact
IPTS Studies on ICT for Learning
The Future of Learning
ICT for the Assessment of Key Competences
Digital Competence
Mainstreaming Creative Classrooms
Open Education in Europe
1to1 Learning initiatives in Europe
Mapping Technologies for Learning
DC Follow up
Where we stand
Multiple Choice
Short Answer
Generation 1Automated administration
and/or scoring of conventional tests
Adaptive TestingGeneration 2 Generation 4
Intelligent Tutors
Automated Feedback
Personalised feedback and
tutoring
Learning 2.0
Generation 3Continuous integrated assessment
Data Mining & Analysis
Behavioural tracking
Generation Re-inventionTransformative strategy: use technology to
change test formats
Peer assessment
Collaborative multimedia learning environments
Online collaborationVirtual laboratories
Games
Simulations
Virtual Worlds
Replication of complex real life situations within a
test
Bun
ders
on, I
nouy
e &
Ols
en (
1989
). T
he f
our
gene
rati
ons
of c
ompu
teri
zed
educ
atio
nal
mea
sure
men
t. In
R. L
. Lin
n (E
d.),
Edu
cati
onal
mea
sure
men
t (T
hird
ed.
, pp.
367
-407
). N
ew
Yor
k: M
acm
illa
nB
enne
tt, R
. E. (
2010
). T
echn
olog
y fo
r L
arge
-Sca
le A
sses
smen
t. In
P. P
eter
son,
E. B
aker
&
B. M
cGaw
(E
ds.)
, Int
erna
tion
al E
ncyc
lope
dia
of E
duca
tion
(3r
d ed
., V
ol. 8
, pp.
48-
55).
fo
rd: E
lsev
ier.
1990 1995 2000 2005 2010 2015 2020 2025Efficient testing Personalised learning
Learning Analytics
Computer-Based Assessment (CBA) Technology-enhanced learning
Current use of ICT in assessment in schools
Central recommendations on the use of ICT in pupil assessment in compulsory education in primary and general secondary education (ISCED 1, 2, 3), 2009/10Source: Eurydice (2011). Key Data on Learning and Innovation through ICT at School in Europe 2011, Education, Audiovisual and Culture Executive Agency.
There is scope for improvement
1st Generation CBA: Problem-Solving
http://www.worldclassarena.org/v5/flash/09_year_old/backtofront.swf
World Class Arena Sample Test: Problem Solving (9 years)
Examples
http://www.worldclassarena.org/v5/flash/13_year_old/pyramids.swf
Problem Solving 13 years
Generation Re-Invention: Problem-Solving
Examples
1st Generation CBA: Situational Judgement
EPSO sample situational judgement test questions; http://europa.eu/epso/discover/prepa_test/sample_test/index_en.htm
Examples
Generation Re-Invention: e-tray exercise
EPS
O s
ampl
e as
sess
men
t cen
tre
e-tr
ay e
xerc
ise
http
://eu
ropa
.eu/
epso
/dis
cove
r/pr
epa_
test
/sam
ple_
test
/inde
x_en
.htm
Examples
Future eAssessment Strategies
Multiple Choice
Short Answer
Generation 1Automated administration
and/or scoring of conventional tests
Adaptive TestingGeneration 2
Generation 4
Intelligent Tutors
Automated Feedback Personalised feedback and
tutoring
Learning 2.0
Generation 3Continuous integrated assessment
Data Mining & Analysis
Behavioural tracking
Generation Re-inventionTransformative strategy: use technology to
change test formats
Computer-Based Assessment (CBA) Technology-enhanced learning
Peer assessment
Collaborative multimedia learning environments
Online collaborationVirtual laboratories
Games
Simulations
Virtual WorldsReplication of complex
real life situations within a test
Bun
ders
on, I
nouy
e &
Ols
en (
1989
). T
he f
our
gene
rati
ons
of c
ompu
teri
zed
educ
atio
nal
mea
sure
men
t. In
R. L
. Lin
n (E
d.),
Edu
cati
onal
mea
sure
men
t (T
hird
ed.
, pp.
367
-407
). N
ew
Yor
k: M
acm
illa
nB
enne
tt, R
. E. (
2010
). T
echn
olog
y fo
r L
arge
-Sca
le A
sses
smen
t. In
P. P
eter
son,
E. B
aker
&
B. M
cGaw
(E
ds.)
, Int
erna
tion
al E
ncyc
lope
dia
of E
duca
tion
(3r
d ed
., V
ol. 8
, pp.
48-
55).
fo
rd: E
lsev
ier.
Learning Analytics
1990 1995 2000 2005 2010 2015 2020 2025Efficient testing Personalised learning
Towards Generation 3: Datamining
Signalshttp://www.itap.purdue.edu/tlt/signals/signals_final/index.htm
Similar: GPS at Northern Arizona University
http://resourceconnect.nau.edu/grade-performance-status.asp
Examples
Towards Generation 3: Behaviour Tracking
Example: SNAPPSNAPP is a software tool that allows users to visualize the network of interactions resulting from discussion forum posts and replies. Teachers can rapidly identify patterns of user behaviour. SNAPP can extract all user interactions from various commercial and open source learning management systems, such as BlackBoard and Moodle.
Examples
Towards Gen 4: Intelligent Tutoring Systems
GnuTutor (http://gnututor.com/)
OpenSource version of AutoTutorhttps://www.youtube.com/watch?v=k6HDEEnYuIA&feature=player_embedded
AutoTutorhttps://www.youtube.com/watch?v=aPcoZPjL2G8
Examples
Generation 1 Generation 2
Generation 4Personalised feedback and
tutoring
Learning 2.0
Generation 3Continuous integrated assessment
Generation Re-invention
Computer-Based Assessment (CBA) Technology-enhanced learning
Collaborative multimedia learning environments
Bun
ders
on, I
nouy
e &
Ols
en (
1989
). T
he f
our
gene
rati
ons
of c
ompu
teri
zed
educ
atio
nal
mea
sure
men
t. In
R. L
. Lin
n (E
d.),
Edu
cati
onal
mea
sure
men
t (T
hird
ed.
, pp.
367
-407
). N
ew
Yor
k: M
acm
illa
nB
enne
tt, R
. E. (
2010
). T
echn
olog
y fo
r L
arge
-Sca
le A
sses
smen
t. In
P. P
eter
son,
E. B
aker
&
B. M
cGaw
(E
ds.)
, Int
erna
tion
al E
ncyc
lope
dia
of E
duca
tion
(3r
d ed
., V
ol. 8
, pp.
48-
55).
fo
rd: E
lsev
ier.
- mainstream use for high stakes testing- further development of innovative test formats- increased ease of use at school level - trend towards automated scoring
→ cultural change: increased acceptance of CBA
Automated administration and/or scoring of conventional tests
Adaptive Testing
Replication of complex real life situations within a test
- Multimedia collaborative learning environments support key competence
development.
- Peer and portfolio assessment evolve naturally.
-Data mining and analysis will make integrated assessment and tutoring possible
in the future.We are in a transitional period: Moving from the “Testing Paradigm” to the (future) “Tutoring Paradigm” in which explicit
testing could become obsolete
Technological trends1990 1995 2000 2005 2010 2015 2020 2025
Efficient testing Personalised learning
Key Questions
1. Which ICT tools, formats, environments, devices etc. are most suitable for the assessment of which Key Competences?
2. How can ICT be used to support competence-based assessment strategies that adequately capture the skills and attitudes dimensions of all eight Key Competences and their transversal nature?
Assessing Key Competences
F = Formative AssessmentD = Diagnostic AssessmentS = Summative Assessment
1 Communication in the mother tongue DS FD FS
2 Communication in foreign languages DS FS
3a Mathematical competence DS FD FDS FDS
3b Basic competences in science and technology FS FDS
4 Digital competence FDS
5 Learning to learn FS FDS
6 Social and civic competences FS
7 Sense of initiative and entrepreneurship FS
8 Cultural awareness and expression FS FS
CB
A
ePor
tfol
ios
Pee
r A
sses
smen
t
Sel
f A
sses
smen
t
Qu
izze
s /
sim
ple
ga
mes
Vir
tual
wor
lds
gam
es
Sim
ula
tion
s
Inte
llig
ent
Tu
tors
etc
.
Focus Areas
•PART 1
•INSPIRATION
Mother Tongue Literacy1 Communication in the mother tongue CBA (DS) Quizzes / simple games (FD) ePortfolios (FS)
Matching spoken and written languagehttp://www.familylearning.org.uk/sight_word_games.html
Foreign language learning
2 Communication in foreign languages CBA (DS) Quizzes / simple games (FD) ePortfolios (FS)
http
://w
ww
.nic
kjr.
de/g
ames
/107
-sw
iper
s-bu
chst
abie
r-bu
ch
Mathematical games3a Mathematical competence CBA (DS) Quizzes / simple games (FD) Simulations (FDS) Inteligent tutoring systems (FDS)
Mathematical games using simulation for secondary school educationSource: http://hotmath.com/games.html.
5 Learning to Learn CBA (F) Peer Assessment (FS) Self Assessment (FS)
http://spark.uts.edu.au/
Example: SparkPlus
Peer Assessment for Collaborative Work
• http://www.youtube.com/watch?v=Tp8v2UBbmN4
www.moodle.org
Moodle is a free web application that educators can use to create effective online
learning sites.
Moodle StatisticsRegistered sites: 77,213Countries: 229Courses: 7,116,466Users: 66,349,750Teachers: 1,292,731Enrolments: 56,306,274Forum posts: 118,413,272Resources: 63,891,942Quiz questions: 147,057,434
Peer Assessment using moodle
ePortfolios
1 Communication in the mother tongue CBA (DS) Quizzes / simple games (FD) ePortfolios (FS)
2 Communication in foreign languages CBA (DS) Quizzes / simple games (FD) ePortfolios (FS)
8 Cultural Awareness and expression ePortfolios (FS) Virtual worlds / games (FS)http://sites.google.com/site/eportfolioapps/overview
/model-k-12
ePortfolios
ePortfolios1 Communication in the mother tongue CBA (DS) Quizzes / simple games (FD) ePortfolios (FS)
2 Communication in foreign languages CBA (DS) Quizzes / simple games (FD) ePortfolios (FS)
8 Cultural Awareness and expression ePortfolios (FS) Virtual worlds / games (FS)
http://sites.google.com/site/eportfolioapps/overview
/levels
Level 1 - ePortfolio as Storage Collection: Creating the Digital Archive (regularly – weekly/monthly)A Focus on Contents & Digital Conversion
Level 2 - ePortfolio as Workspace Collection + Reflection: Immediate Reflection on Learning & Artifacts in Collection (regularly) A Focus on Process & Documentation of Learning
Level 3 - ePortfolio as Showcase Selection/Reflection/Direction (each semester? End of year?)A Focus on Product & Documentation of Achievement
→ A (formative/summative) assessment approach on the basis of artefacts → Focus on production and reflection
A game approach to arts
8 Cultural Awareness and expression ePortfolios (FS) Virtual worlds / games (FS)
http://pheon.org; http://pheon.org/intro.html; http://americanart.si.edu/multimedia/games/pheon.cfm.
Pheon is a multimedia scavenger hunt that tests players’ ingenuity through exploring the collection of the Smithsonian American Art Museum, making objects, and texting through the galleries
ePortfolios meetVirtual reality
Science: Molecular Workbench
3b Basic competences in science and technology CBA (S) Virtual Worlds / games (FS) Simulations (FDS)
http://mw2.concord.org/public/part2/diffat/page3.cml
3a Mathematical competence CBA (DS) Quizzes / simple games (FD) Simulations (FDS) Inteligent tutoring systems (FDS)
Simulations for mathematics
Sample task from US future New State Tests
(2014/15)
http://www.k12center.org/rsc/pdf/a-sea-change-in-assessment-letter-size.pdf
Cognitive Tutor (Carnegie)
3a Mathematical competence CBA (DS) Quizzes / simple games (FD) Simulations (FDS) Inteligent tutoring systems (FDS)
http://www.youtube.com/watch?v=RR4HPrGlUz8&feature=related
Science: Quest Atlantis
3b Basic competences in science and technology CBA (S) Virtual Worlds / games (FS) Simulations (FDS)
Example: Quest Alantis
Quest Atlantis (QA) is an international learning and teaching project that uses a 3D multi-user environment to immerse children, ages 9-16, in educational tasks. QA combines strategies used in the commercial gaming environment with lessons from educational research on learning and motivation.
Citizenship6 Social and civic competences CBA (DS) Peer/Self Assessment (FDS) Virtual Games / Simulations (FDS)
Educational game, designed to help teach concepts in citizenship, geography, and media. Developed by Serious Games International, it has detailed lesson plans and assignments for students.
Example: Child labour in Bangladesh:
http://www.youtube.com/watch?v=mf9Ju86IVYw
Global Conflicts http://www.globalconflicts.eu/
Examples
•PART 2
•CRITICAL INSPECTION
Digital Competence
4 Digital competence CBA (S) Simulations (FDS)
Internet and Communication Core Certification (IC3) exam sample questionhttp://www.certiport.com/Portal/common/htmllibrary/Sales-Video-Demos/demo_ic3.html
→ Focus on knowledge
CBA forDigital Competence
Digital Competence: IC3
4 Digital competence CBA (S) Simulations (FDS)
Inte
rnet
and
Com
mun
icat
ion
Cor
e C
erti
fica
tion
(IC
3) e
xam
sa
mpl
e qu
esti
onht
tp:/
/ww
w.c
erti
port
.com
/Por
tal/
com
mon
/htm
llib
rary
/Sal
es-
Vid
eo-D
emos
/dem
o_ic
3.ht
ml
→ Focus on operational skills
CBA forDigital Competence
Entrepreneurship Questionnaire
7 Sense of initiative and entrepreneurship CBA (DS) Peer/Self Assessment (FDS) Virtual Games / Simulations (FDS)
?https://
www.careeringahead.com.au/survey/Entrepreneurship/
Generic/questions.asp
→ Focus on personality/disposition as a static fact
Entrepreneurship Questionnaire
Conclusions
ICT offers powerful, engaging assessment formats, in particular:
CBA ePortfoliosSelf / Peer
AssessmentQuizzes /
simple games Virtual worlds/ gamesSimulations
Basic literacy(foreign/mother tongue)
Basic mathematical competence
Learning to Learn
Creative writing(foreign/mother tongue)
Cultural awareness and expression
Scientific competence
Digital competence
Social and civic competences
Educational software
Mathematical competence
CBA lends itself to simplified, knowledge-based assessment→ Pedagogy comes first! Careful selection of question formats
ICT Assessment approaches are underdeveloped for:
Learning to Learn Sense of initiative and entrepreneurship Social and civic competences
Caveat:
The Paradigm Shift
Technology
1990 1995 2000 2005 2010 2015 2020 2025Focus on: Efficient testing Focus on: Personalised learningEra of Computer-Based Assessment (CBA) Era of Technology-enhanced learning
PedagogyEfficient Testing
Validity of Scores
QuickFeedback
Authentic Contexts
CBA(Gen.1)
AdaptiveTests
(Gen.2)
AutomaticScoring
Gen. Re-Invention
Learning2.0
Immersive Learning
LearningAnalytics
IntelligentTutors
? ? ? ?
→ Primacy of Pedagogy: Pedagogical guidelines for intergrated
assessment
• Thank you very much for your attention!
• Christine Redecker• Yves Punie
• [email protected]• [email protected]
http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=5719
further information ……
http://is.jrc.ec.europa.eu
Newsletters: http://is.jrc.es/pages/newsletter.html
Publications: http://is.jrc.ec.europa.eu/pages/Publications.html
Contact: [email protected]