Madrid April 15-17, 2013 European Commission Information Society and Media General Assembly GaLA Game and Learning Alliance The European Network of Excellence on Serious Games SIG3.5 – SGs for Humanities & Heritage user studies Michela Mortara, Chiara Eva Catalano, CNR – IMATI Jeffrey Earp, CNR-ITD
18
Embed
SIG3.5 – SGs for Humanities & Heritage user studies
SIG3.5 – SGs for Humanities & Heritage user studies. Michela Mortara , Chiara Eva Catalano, CNR – IMATI Jeffrey Earp, CNR-ITD. Overview. Icura [ ongoing , 50%]. Playing History – The Plague [started, 10%]. Playing History – The Plague [started, 20%]. User study #1: Icura. - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
GaLAGame and Learning Alliance
The European Network of Excellence on Serious Games
SIG3.5 – SGs for Humanities & Heritageuser studies
Michela Mortara, Chiara Eva Catalano, CNR – IMATIJeffrey Earp, CNR-ITD
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Overview
Icura [ongoing, 50%]
• Playing History – The Plague [started, 20%]
Playing History – The Plague [started, 10%]
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
USER STUDY #1: ICURA
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Game description
• Educational objective: cultural awareness - Japanese etiquette
• Genre: adventure• Environment: 3D • Platform: PC• Intended educational context: informal (at home)• Developer: TU-Wien• Duration: less than 1 h• Target: secondary school and higher• Previous studies: evaluation done by the developers at
the Wien university on a group of 20 students. Reports on papers and PhD. thesis
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Previous evaluation
• Pre-questionnaire (demographic data & game habits)• Pre-test with 12 questions ( on language basics, behaviour
& etiquette, culture and society)• Game session• Post-test (same 12 questions)• Final questionnaire about user satisfactionObservations:• On average from +5 to +10 correct answers btw pre and
post– Selective attention bias
• Learning gain linked to the way the content is communicated (information agent vs quest)
• Lacking estimate of retention
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Our setting
• Same phases but different questions in the pre-test to avoid the selective bias
• Higher participant number (CNR, RWTH, ORT, DIBE)• Open answers to assess higher level in Bloom’s
taxonomy (post questionnaire) e.g.:• What have you learnt from the game and how (through what mechanics)? • Did the game increase your interest towards Japan/ Japanese culture?
Would you read more about Japan? Would you visit it?• How would you summarise Japanese culture in 3 adjectives?• What can you deduce about Japanese culture from the facts you learnt in
the game? • How would you compare Japanese and Western culture?
• Plan to repeat the post test after a few months to evaluate retention
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Current status
• CNR session– 10 university students– Colleagues&friends
• RWTH session– 17 university students
• ORT session– 12 people– students
• DIBE session
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Partial outcomes
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Partial outcomes
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Partial outcomes
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
PLAYING HISTORY – THE PLAGUE
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
The game
• Educational objective: history, historical reconstruction • Genre: adventure• Environment: 3D • Platform: browser (Unity Web player)• Intended educational context: formal (at school)• Developer: Serious Games Interactive• Duration: nearly 3 hours• Target: 8-13• Previous studies: evaluation done by a master student
with two primary school classes in Denmark. Report in her Master thesis (in Danish).
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
The experience: a Joint Research Action
• JRA involving CNR-IMATI (SIG3.5), CNR-ITD (TC2.8, T6.3), SGI and DIS (Deledda International School) in Genova
• Objectives:– explore the suitability and effectiveness of serious games in the
field of humanities and cultural heritage in schools;– evaluate the educational efficacy of the game and the level of
student engagement and enjoyment triggered by the learning activities;
– investigate critical aspects of SG deployment in formal ed./school contexts, especially teacher’s role
– examine cultural/educational issues related to game’s adoption in Italy (game produced in northern Europe and set in medieval Italy)
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
The context
• School: English-speaking school in Genoa following International Baccalaureate & IB Middle Years programmes
• Classes: 2 classes Year 7 (11-12 year olds). Approx. 35 pupils
• Deployment settings: computer lab, classroom, home access
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Research framework
• Approach: qualitative, design based research – DBR Collective
• Tools:
– question matrix for gathering teachers’ GBL/ICT experience & attitudes, plus their expectations, intentions and goals for the experience
– pedagogical planning tool (teachers document intentions/goals of intervention and plan learning activity sequence)
– researchers’ observation sheets for monitoring classroom activities
– pupils’ pre/post tests (in teacher’s resource pack from SGI)
– mind map recording pupils’ collective pre/post knowledge/awareness
– teachers’ monitored focus group
– teacher questionnaire
– reflection sheet for recording researchers’ attitudes, intentions, goals and interventions in the experience
1. Teacher preparation– introduction to the experiment, the game and its teaching resource pack– teachers’ group discussion of matrix questions (researcher compiled)– tech. training session (game’s teacher functions; ITD pedagogical
(making collective mind map)– class briefing: game presentation + play– gameplay sessions in pairs (plus possible home play)– pupil summative test & debriefing (re-examining collective mind map)
= total 4/5 school periods with researcher monitoring & support
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
… Activity phases 3 - 6
3. Teacher debriefing– focus group (inc. re-examination of matrix questions and pre-
deployment responses, and of pedagogical plan/s)– final questionnaire
4. Analysis of data gathered from phases 1-3
5. Follow up – Repeat of post test to evaluate pupil retention
6. Final data analysis and reporting
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Current status
• Series of organisational meetings held involving CNR, DIS & SGI
• Activity phase 1 commenced– introduction to teachers– discussion / compiled of teachers’ of question matrix
Issues with running game on DIS network have delayed commencement of deployment sessions with pupils• Solution: DIS agreed to move students to CNR (2