Games and assessment week 6

Games and Assessment

Week 6

Asking the Right Question

Effective systematic inquiry processes begin with an appropriate question.

Asking the right question is often the most difficult part of assessment.

Learning Outcome related questions

• Start with a specific, measurable, time-bound learning outcome.

• Example: After unit 3, students will be able to list the 4 fun keys to create important emotions.

Learning Outcome related questions

• Example: After unit 3, students will be able to list the 4 fun keys to create important emotions.

Answer: The 4 Fun Keys create games' four most

important emotions:

1. Hard Fun: Fiero - in the moment personal triumph over adversity

2. Easy Fun: Curiosity

3. Serious Fun: Relaxation and excitement

4. People Fun: Amusement

Traditional methods for data collection:

• Exams, online or in class

• Quizzes, online or in class

• Use grading rubrics

Discussion Question: How could you collect this information using games or other non-traditional methods?

Process Evaluation related questions

• Do our processes work well? Could they work better?

• Do our policies help or hinder student success?

• To what extend is our program being implemented according to plan?

• Do we have the resources we need?

• What are the barriers or challenges to implementing our processes?

• How well did students understand our process?

• Are faculty/instructors sufficiently trained?

• Are the directions clear?

Traditional methods for data collection:

• Follow-up survey

• Point-of-service interview (catch student right after they created an account on the game site…ask for their experiences)

• Secret shopper (have a neutral third party go through the process and give you feedback)

Discussion Question: How could you collect this information using games or other non-traditional methods?

Net Effect related questions

• What is the effect of completing 30-credits online at FRCC?

• Can the net effects be attributed to the course games versus maturation?

Traditional methods for data collection:

• Longitudinal tracking systems

• Set up experiments or quasi-experiments

• Ex post facto (after the fact)/post-experience surveys

Discussion Question: How could you collect this information using games or other non-traditional methods?

Goal Based questions

These are based on S.M.A.R.T. (Specific, Measurable, Attainable,

Realistic, Time-bound) goals.

Example question(s):

• Did we meet or exceed our goal?

• If not, what could we have done differently?

• Why weren’t people signing up?

• Were we being unrealistic?

Traditional methods for data collection:

• Institutional Research

• Tracking systems you have purchased or developed (count the number of students who signed up)

• Further qualitative methods (surveys, interviews, focus groups, etc.) to understand the reasons why or why not.

Discussion Question: How could you collect this information using games or other non-traditional methods?

Comparison questions

Examples:

• Do students who played Game A perform better on the final exam than those who played Game B?

• How much would students practice if they practiced on the game FreeRice versus a quiz for points?

Traditional methods for data collection:

Quasi-experimental or Ex Post Facto research methods

Discussion Question: How could you collect this information using games or other non-traditional methods?

Satisfaction related questions

Example: How satisfied are students with ______________?

Traditional methods for data collection:

• Satisfaction Surveys

• Focus groups

• Interviews

Discussion Question: How could you collect this information using games or other non-traditional methods?

Who Are Our Students questions

Example:

What type of student prefers using a game to learn versus other modes?

Traditional methods for data collection:

• Focus groups• Interviews

• Surveys– Paper-pencil

– Electronic

– Smart phones (surveys can now be done via smart phone)

Discussion Question: How could you collect this information using games or other non-traditional methods?

What Are Our Students’ Experiences questions

Examples:

• What do students view as engaging in an online course?

• HOW did the game help students learn?

• What is it about the game that enhanced learning?

Traditional methods for data collection:

• Think-alouds (sit with a student while he/she is working a problem or playing a game, have the student talk aloud while going through the process)

Discussion Question: How could you collect this information using games or other non-traditional methods?

Game Component questions

Examples:

What features should my game include in order to engage students?

Discussion Question: What are other game related questions?

MAJOR DISCUSSION QUESTION

• How would YOU evaluate this MOOC?

• What different types of questions would you ask?

• How would you collect data to help answer each question?