Assessment of Strategies in the Diffusion Simulation Game

Assessment of Strategies in the Diffusion Simulation Game

Jake [email protected]

Rod [email protected]

Instructional Systems TechnologyIndiana University - Bloomington

Research Goal

Analyze gameplay data to assess the fidelity of the game with the theory it is designed to teach.

2

Key Concepts

Diffusion of Innovations Theory

DSG - Simulation game for learning

Fidelity of simulations

Analysis of Patterns in Time (APT)

3

Diffusion of Innovations Theory

Everett Rogers

Source: http://www.12manage.com/methods_rogers_innovation_adoption_curve.html

4

Demo of DSG new version (beta)

5

6Current Version ~ Detailed View

Predicted Successful Strategies

1. Get to know the staff 2. Take a client-oriented approach3. Utilize opinion leaders4. Provide opportunities for staff to evaluate

the innovation5. Select mass media communication

channels to raise awareness of the staff early in the game.

6. Utilize interpersonal channels7. Utilize early adopters

7

Data Collection

8

Game typeNumber of games

Number of turns M SD

Successful 341 11276 33.07 5.791

Unsuccessful 1532 73941 48.26 7.299

Undetermined 488 22077 45.24 6.728

Total 2361 107294 45.44 8.720

• 10,000 game sessions from 2006-2009• 2361 Completed Games• 341 Successful Games (all 22 adopters)• 1532 Unsuccessful Games (<16 adopters)

Data Analysis

Compared mean relative frequencies between Successful and Unsuccessful groups

Used APT to look at sequential and joint occurrences in gameplay patterns

9

Findings & Discussion

Research Question 1: What strategies were successful in the game sessions being studied?

Most frequently used activities

Most frequent sequences of activities

Most frequent joint occurrences

(activity + targeted faculty)10

Findings & Discussion

Successful games•Print (5 : 1)

•Site Visit (3 : 1)

•Local Mass Media (2 : 1)

•Talk To Talk To

•Print Print

•Print Print Print Print

•Opinion Leaders: Demonstration (1.5 : 1) 11

Unsuccessful games•Ask Help (5 : 1)

•Pilot Test (5 : 1)

•Talk To Talk To

•Talk To Ask Help

•Talk To Talk To Talk To Talk To

•Principal: Site Visit (9 : 1), Ask Help (2 : 1)

Findings & Discussion

Research Question 2: Are the strategies that were successful within the game aligned with strategies that would be predicted by the diffusion of innovation theory?

Only 3 of the 7 predicted strategies were employed more frequently in successful games.

12

Findings & Discussion

<<< APPARENTLY >>>1. Get to know the staff 2. Take a client-oriented approach3. Utilize opinion leaders4. Provide opportunities for staff to evaluate

the innovation5. Select mass media communication

channels to raise awareness of the staff early in the game.

6. Utilize interpersonal channels7. Utilize early adopters

13

Findings & Discussion

Research Question 3: Do other successful strategies exist that do not relate to theories of the diffusion of innovation?

Targeting Laggards

Extreme winners (won in less than 26 turns)• Frequency of activity use• Sequences• Joint Occurrences

14

Limitations

Extant Data (Adoption Stage, player id)

Relative frequencies as metric

15

Future Research

Follow up study with additional data collected

Designing a data collection method to support APT (data collected at a granular level) and developing a pattern query tool

Use of instructional strategies to support learning and address misconceptions (such as debriefing)

16

References DIFFUSION SIMULATION GAME. (1976). Molenda, M., &

Young, P. Bloomington, IN: Indiana University, 201 North Rose Avenue, Bloomington, IN 47405-1006.

Frick, T. W. (1990). Analysis of patterns in time (APT): A method of recording and quantifying temporal relations in education. American Educational Research Journal, 27(1), 180-204.

Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: The Free Press.

17

You can play too!

Production version v.1:http://www.indiana.edu/~istdemo

New version (beta):http://www.indiana.edu/~simgame

18