Richard Wainess Dissertation v.4Presented 12/09/05 The Effect of Navigation Maps on Problem Solving Tasks Instantiated in a Computer-Based Video Game Committee.

Richard Wainess Dissertation v.4 Presented 12/09/05

The Effect of Navigation Maps on Problem SolvingTasks Instantiated in a Computer-Based Video Game

Committee Members: Dr. Harold O’Neil (Chair)

Dr. Edward Kazlauskas

Dr. Yanis Yortsos (Outside Member)

Richard WainessDissertation Presented to the

Faculty of the Graduate School

University of Southern CaliforniaDecember 9, 2005

Slide 1 of 14


Research HypothesesProblem Solving

– Hypothesis 1: Participants who use a navigation map (the treatment group) will exhibit significantly greater content understanding than participants who do not use a navigation map (the control group).

– Hypothesis 2: Participants who use a navigation map (the treatment group) will exhibit greater problem solving strategy retention than participants who do not use a navigation map (the control group).

– Hypothesis 3: Participants who use a navigation map (the treatment group) will exhibit greater problem solving strategy transfer than participants who do not use a navigation map (the control group).

– Hypothesis 4: There will be no significant difference in self-regulation between the navigation map group (the treatment group) and the control group. However, it is expected that higher levels of self-regulation will be associated with better performance.

Motivation– Hypothesis 5: Participants who use a navigation map (the treatment group) will

exhibit a greater amount of continuing motivation, as indicated by continued optional game play, than participants who do not use a navigation map (the control group).

Slide 2 of 14


Cognitive Load Theory• Cognitive Architecture

– Auditory/Verbal and Visual/Spatial Channels/Memory(Baddeley,1986; Mayer & Moreno, 2003)

– Limited Working Memory (Brunken et al., 2003)• 7 +/- 2 (Miller, 1956)• Possibly only 2 or 3 novel elements (Paas et al., 2003)

– Unlimited Long-Term Memory (Mousavi et al., 1995)

• Cognitive Load = mental capacity imposed on working memory (Sweller & Chandler, 1994)

– Controlled by schema development & Automation– Intrinsic Cognitive Load: Necessary (Brunken et al., 2003; Paas et al., 2003).

– Germane Cognitive Load: Required to process intrinsic load (Renkl & Atkinson, 2003).

– Extraneous Cognitive Load: Unnecessary stimuli (Brunken et al. 2003).• Seductive Details (Mayer et al., 2001; Schraw, 1998).

Slide 3 of 14


Scaffolding(including Graphical Scaffolding)

• Instructional methods– Should keep cognitive load low (Clark, 2003)

– External methods which replace internal processes (Clark, 2001)

• Scaffolding is an instructional method

• Scaffolding provides support during learning(Allen, 1997; Chalmers, 2003; van Merrienboer et al., 2002, 2003)

• Graphical Scaffolding– Includes maps and menus as advance organizers (Jones et al., 1995)

– Maps supported by researchers as visual aids and organizers(Benbasat & Todd, 1993; Chou & Lin, 1998; Ruddle et al., 1999)

– Should be used for visual tasks (Mayer et al., 2002)

Slide 4 of 14


Navigation Maps

• Navigation: Tracking one’s position in an environment to arrive at a destination (Cutmore et al., 2000)

• Occlusion: when a path is blocked visually (Cutmore et al., 2000)

• Navigation maps effective for occluded 3-D navigation(Cutmore et al. 2000; Dempsey, 2002)

• Navigation maps effective in 2-D environments with complex problem solving tasks (Baylor, 2001; Chou & Lin, 1998; Chou et al., 2000)

• Navigation maps effective in 3-D occluded environments with simple problem solving tasks (Galimberti, 2001)

Not yet examinedNavigation in 3D, occluded, environments

with complex problem solving tasks

Slide 5 of 14


Navigation Map

Floor Plan of Mansion’s First Floor from SafeCracker®

Slide 6 of 14


Method: Design

• Design– Experimental, Posttest Only, 2x2 Repeated Measures

• First factor—two levels (one treatment group, one control group)

• Second factor—two levels (occasion 1, occasion 2)

– Random assignment to treatment group or control group

– Treatment receives navigation map; Control Group doesn’t

– Pilot study and main study

– Approx. 90 minutes

Slide 7 of 14


Method: Participants

• Participants (paid $15 for participation)

– Main Study (November 11, 2004 through March 21, 2005)

• 71 English-speaking males and females– Data analyzed for 64 (33 treatment, 31 control)

• Undergraduate and graduate students at USC• No prior experience with the game

SafeCracker®• Average Age: a few days short of 29 years

Slide 8 of 14


Method: Timing Chart for Main Study

Introduction and study paperwork

Self-regulation and demographic questionnaires

Introduction to knowledge mapping software

Introduction to SafeCracker

Introduction to map reading for the treatment group

FIRST GAME (3 rooms) plus task completion form

Knowledge map creation (occasion 1)

Problem solving strategy retention and transfer questionnaire (occasion 1)

SECOND GAME (3 rooms) plus task completion form

Knowledge map creation (occasion 2)

Problem solving strategy retention and transfer questionnaire (occasion 2)

Debriefing

Optional additional playing time

Slide 9 of 14


ResultsProblem Solving

– Hypothesis 1: Participants who use a navigation map (the treatment group) will exhibit significantly greater content understanding than participants who do not use a navigation map (the control group).NOT SUPPORTED

– Hypothesis 2: Participants who use a navigation map (the treatment group) will exhibit greater problem solving strategy retention than participants who do not use a navigation map (the control group).NOT SUPPORTED

– Hypothesis 3: Participants who use a navigation map (the treatment group) will exhibit greater problem solving strategy transfer than participants who do not use a navigation map (the control group).NOT SUPPORTED

Motivation– Hypothesis 5: Participants who use a navigation map (the treatment

group) will exhibit a greater amount of continuing motivation, as indicated by continued optional game play, than participants who do not use a navigation map (the control group).NOT SUPPORTED

Slide 10 of 14


Results (cont’d)

Problem Solving– Hypothesis 4: There will be no significant difference in self-regulation

between the navigation map group (the treatment group) and the control group. However, it is expected that higher levels of self-regulation will be associated with better performance.

– PARTIALLY SUPPORTED• Navigation Map Group: Positive correlation between mental effort

and problem solving retention improvement

• Control Group: Negative correlation between planning and problem solving transfer improvement

• Self-efficacy not supported

• Monitoring not supported

Slide 11 of 14


Discussion

• Reduction of performance for treatment group offsetting effect of navigation map– Contiguity Effect (Mayer et al., 1999; Mayer & Moreno, 2003; Mayer & Sims,

1994; Moreno & Mayer, 1999)

• Spatial and temporal• Results in Split Attention Effect (Atkinson et al., 2000; Mayer, 2001;

Tarmizi & Sweller, 1998)– Cognitive strain during integration of information

– Extraneous Load (Harp & Mayer, 1998; Mayer, Heiser, & Lonn, 2001; Moreno & Mayer, 2000; Renkl & Atkinson, 2003; Schraw, 1998)

• Seductive Details (Mayer et al., 2001; Schraw, 1998)– Affects retention and transfer (Moreno & Mayer, 2000)

• Navigation Map?

Slide 12 of 14


Discussion (cont’d)

• Increase in performance for both groups, offsetting effect of navigation map– Strategy Priming

• Ready mind to allow or engage particular relevant schema (http://filebox.vt.edu/8080/users/dereese2/module8/module08bkup/IDProjectWebpage/lesson4.htm)

• Repetition closely aligned with skill acquisition (Dennis & Schmidt, 2003)

– Lack of priming reduces process of selecting (Moreno & Mayer, 2005)

• Primed during:– Knowledge map training and SafeCracker training

– During navigation map and basic navigation training

– Start of each game

Slide 13 of 14


Implications

• Navigation maps do not guarantee improved performance

• Further studies are suggested– Manipulation of contiguity effect

• Integrated versus separated map

– Manipulation of extraneous details• Low versus high levels of environmental fidelity (both with map use)

• Small versus large environment (both with map use)

Slide 14 of 14


BACK UP

Slide 15


Problem Solving Assessment

• Based on O’Neil’s (1999) Problem Solving Model– Requires content understanding, problem solving strategies, self-

regulation

• Transfer questions are alternative to transfer tasks (Moreno & Mayer, 1998)

• Declarative knowledge measured by retention (Day et al., 2001)

• Knowledge map: concepts and links (Schau & Mattern, 1997)

– Reflects the organization of knowledge (Day et al., 2001)

– Reliable and efficient measure of Content Understanding(Herl et al., 1999; O’Neil, 1999; Ruiz-Primo et al., 1997; Schacter et al., 1999)

– Predictive of both retention and transfer (Day et al., 2001)

– Reliable measure of Problem Solving Strategies (Baker & Mayer, 1999)

Slide 16


Problem Solving Assessment (cont’d)

• Measuring Problem Solving Strategies– Domain general and specific (Alexander, 1992; Bruning et al., 1999)

– Knowledge Mapping (Baker & Mayer, 1999)

– Problem solving questions• Positively correlated with retention and transfer (Mayer & Baker, 1998)

• Measuring Self-Regulation– Includes Metacognition & Self-efficacy (O’Neil, 1999)

• Metacognition: planning & self-checking (Pintrich & DeGroot, 1990)• Self-efficacy: mental effort and self-efficacy (Zimmerman, 1994, 2000)

– Trait self-regulation questionnaire (Hong & O’Neil, 2001)• Includes planning, self-checking, mental effort, and self-efficacy• 32 questions: 8 for each of the four sub-categories

Slide 17


Games and Simulations

• Games: rules, constraints/privileges, imaginative, linear (Gredler, 1996)

• Simulation-Games: combination of games and simulations (Gredler, 1996)

• Motivation in games: fantasy, control & manipulation, challenge & complexity, curiosity, competition, feedback, fun

• Positive outcomes:– Numerous knowledge outcomes attributed to games and simulations

• Warning about anecdotal and descriptive evaluations (Leemkuil et al., 2003; Wolfe, 1997)– Generalizable skills outcomes

(Day et al. 2001; Green & Bavelier, 2003; Greenfield et al., 1994)

• Negative or null outcomes:– Reviews and meta-analyses cite mixed or negative reviews

(Dekkers & Donatti, 1981; Druckman, 1995)– Positive attitude toward games doesn’t necessarily equal learning

(Brougere, 1999; Salas et al., 1998; Salomon, 1984)

• Outcomes related to Instructional Design, not games/simulations(de Jong & van Joolingen, 1998; Garris et al., 2002; Gredler, 1996; Leemkuil et al., 2003; Thiagarajan, 1998)

Slide 18


O’Neil’s Problem Solving Model

ContentUnderstanding

Problem solvingStrategies

Self-Regulation

DomainSpecific

DomainIndependent

Metacognition Motivation

Planning Self-Monitoring

Effort

Problem Solving

O’Neil (1999)Slide 19


Content Understanding:Knowledge Mapping Software

Slide 20


key

books

crack

room

tool

desk

direction map

safeclue

brochure

part of

used for

contains

used for

causes

results from

contains results from

causes

used for

contains

contains

containsresults from

part ofcontains

uses

causes

Sample SafeCracker® Knowledge Map

contains

Slide 21


Problem Solving StrategyRetention and Transfer Questionnaire

Retention Question:1. List the ways you found rooms and opened safes

Transfer Question:1. List some ways to improve the design of the game

play for opening safes

• 28 idea units generated for Retention Question

• 21 idea units generated for Transfer Question

Slide 22


Self-Regulation Questionnaire

• Based on O’Neil (1999) Problem Solving Model• Trait self-regulation questionnaire (O’Neil & Herl, 1998). • 32 Questions: 8 each of four measures

– planning

– self-checking/monitoring

– self-efficacy

– effort

Slide 23


Data Analysis

• Descriptive statistics: Means, Standard Deviation, etc. for content understanding, problem solving strategy retention and transfer, trait self-regulation, number of safes opened, and continuing motivation.

• Mixed Group, Repeated Measures Factorial ANOVA: Examine effect of use/non-use of navigation map on content understanding, problem solving strategy retention and transfer, and number of safes opened.

• Correlations: between self-regulation components (planning, monitoring, effort, self-efficacy) and content understanding, problem solving strategy retention and transfer, and number of safes opened for each group and both groups combined.

Slide 24

Richard Wainess Dissertation v.4Presented 12/09/05 The Effect of Navigation Maps on Problem Solving Tasks Instantiated in a Computer-Based Video Game Committee.

Documents