Joss Ives - Group Quizzes as an Assessment That Supports Learning (BCAPT AGM2012)

Group quizzes as an assessment that supports learning

[email protected]: http://learnification.wordpress.comTwitter: @jossives

After an exam, students will often hang out in the hallway to discuss the exam

After an exam, students will often hang out in the hallway to discuss the exam

Which equation should I have used for question 3?

After an exam, students will often hang out in the hallway to discuss the exam

I totally nailed number 2. I have got to tell these guys what I did.

After an exam, students will often hang out in the hallway to discuss the exam

Now that I’m saying it out loud, I realize where I went wrong on #1

I see group quizzes as a way to bottle that moment

Now I get it!

My implementation

Strengths & weakneses

My findings

OUTLINE

My implementation(weekly quiz)

The individual quiz has 4 to 6 items and takes 20 minutes to write

1. Multiple-choice questions

2. Ranking tasks

3. Short problems

All quiz questions are converted to multiple-choice and the the group component takes 10 minutes

A! B!

C!

They receive immediate feedback thanks to the scratch cards

$85 / 500 card $0.17 / card

(or chop up the cards with 50 questions each and the cost drops below $0.05 / card)

http://www.epsteineducation.com

[Other methods to provide immediate feedback include LMSs, online homework systems and some clickers]

Their overall quiz grade is 75% from their individual score and 25% from their group score

Strengths&

Weaknesses

Both students benefit when one explains an idea to the other

?

!

The group quiz is the highest level of student engagement I have observed in any of my courses

VS.

Even the shyest students

participate

My students really like the format

Group midterm?Please!!!

[Note: I have used these in my intro calculus-based courses as well as my 3rd-year Quantum Mechanics I course and the feedback from students has been overwhelmingly positive]

My students feel that the group quizzes make a large contribution to their learning?

“How much do you feel that group quizzes have contributed to your learning so far in this course?” [W2012, PHYS112]

A) A large contribution to my learning.B) A small contribution to my learning.C) They don’t contribute to my learning.

Immediate feedback means that they learn the correct answer when they are most receptive

Anecdotally, they are not satisfied when they find the correct answer through guessing

CAN YOU EXPLAIN TO US WHY E IS THE CORRECT ANSWER?

The most obvious weakness is that this format requires extra time

[For people already using quizzes in their courses, the extra 10 minutes required to add the group component is worth whatever 10 minutes needed to come out]

Writing the individual quiz first moderates worries about social loafers receiving free marks

FREEMARKS

(bwahaha)

Scratching tends to be very democratic, which means that the loudest person doesn’t usually dominate

A!

C

C

There is no compelling evidence that answering first individually improves group performance or learning

01020304050607080

I => G(Individual)

I => G (Group) Group First

Perc

enta

ge C

orre

ct (%

)

[C. Singh, AJP 73 (5), May 2005]

[Note: This suggests a lack of evidence for the best practice suggestion for Mazur’s Peer Instruction (and Think-Pair-Share) that it is crucial that the students vote first individually before they have group discussions]

My Findings

The questions are reasonably challenging for the students

Group Questions

Individual Questions

All Questions(n=1008)

57.3 1.6%(n=578)

97.8 0.6%(n=565)

2.2 0.6%(n=13)

42.7 1.6%(n=430)

73.0 2.1%(n=314)

27.0 2.1%(n=116)

Correct individuals are infrequently led astray

Group Questions

Individual Questions

All Questions(n=1008)

57.3 1.6%(n=578)

97.8 0.6%(n=565)

2.2 0.6%(n=13)

42.7 1.6%(n=430)

73.0 2.1%(n=314)

27.0 2.1%(n=116)

The group usually outperforms its strongest individual

71.8%

21.0%7.2%0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

Group > Individual(n=209)

Group = Individual(n=61)

Group < Individual(n=21)

Perc

enta

ge

The results for incorrect individuals are consistent with them being in a group with a correct individual

Group Questions

Individual Questions

All Questions(n=1008)

57.3 1.6%(n=578)

97.8 0.6%(n=565)

2.2 0.6%(n=13)

42.7 1.6%(n=430)

73.0 2.1%(n=314)

27.0 2.1%(n=116)

0

10

20

30

40

50

60

70

Grou

p qu

estio

n co

rrec

t aft

er a

ll in

divi

dual

s wer

e in

corr

ect (

%)

Groups composed of only incorrect individuals show good evidence of learning

Me Singh Me Gilley

[C. Singh, AJP 73 (5), May 2005] [B. Gilley, S. Harris, Poster, GSA 2012]

Half of the time incorrect groups will the find correct answer on their second try

Group Questions

Individual Questions

All Questions(n=1008)

57.3 1.6%(n=578)

97.8 0.6%(n=565)

2.2 0.6%(n=13)

42.7 1.6%(n=430)

73.0 2.1%(n=314)

27.0 2.1%(n=116)

And this is significantly better than chance

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

Correct Incorrect 1 Mistake 2 Mistakes 3 Mistakes 4 Mistakes

Gro

up (

corr

ect a

fter x

mist

akes

)

In summary: group quizzes

1. Learning takes place

2. Students like them

3. Take extra time but worth it

(stay tuned for results from this past year)

Group exams pros from other literature

• Development of collaboration skills [1].• Students enjoy them and there is an increase

in overall enjoyment of course [1].• Promote higher-level thinking [2].• Increase in student retention of information [3] (Note that Ref. 3 did not control for

time on task).• Improved student learning [2,4].

[1] Stearns, S. (1996). Collaborative Exams as Learning Tools. College Teaching, 44, 111–112.[2] Yuretich, R., Khan, S. & Leckie, R. (2001). Active-learning methods to improve student performance and scientific interest in a large introductory oceanography course. Journal of Geoscience Education, 49, 111–119.

[3] Cortright, R.N., Collins, H.L., Rodenbaugh D.W. & DiCarlo, S.T. (2003). Student retention of course content is improved by collaborative-group testing, Advan. Physiol. Edu. 27: 102-108.[4] Gilley, B. & Harris, S. (2010). Group quizzes as a learning experience in an introductory lab, Poster presented at Geological Society of America 2010 Annual Meeting.

Acknowledgement: The references, discussion of group exams pros/cons and some of the analysis borrow heavily from Ref. [4] (thanks Brett!).