PeerWise workshop at NTU Singapore

student generated content for enhanced engagement and learning

Simon Bates [email protected] The University of Bri7sh Columbia

Overview for the session

1) About PeerWise 2) Hands-‐on session 3)  Research highlights 4)  Q&A


1) About PeerWise


a web-‐based MCQ repository created by students

InsFtuFons signing up per year:

2009: 22 2010: 66 2011: 204 2012: 266 2013 (Jan-‐Jun): 214

Growing content repository: Courses: 2,500 Logins/month: 75,000 Ques7ons: 600,000 Answers: 12,000,000

0

5,000,000

10,000,000

15,000,000

20,000,000

25,000,000

30,000,000

Jun 09

Dec 09

Jun 10

Dec 10

Jun 11

Dec 11

Jun 12

Dec 12

Jun 13

Dec 13

Jun 14

Dec 14

May 15

Total a

nswers at end of d

ate

Submitted Answers (2009‐2015)

Student ownership over learning resource

Student familiarity with social soPware

Leveraging student energy and creaFvity

Why student authored ques7ons?

•  A student contribu7ng a new ques7on would develop the ques7on stem:

QuesFon

Student authored ques7on HUBS192, 2010

University of Otago

Why student authored ques7ons? AlternaFves

•  And a set of plausible alterna7ves

•  Of course, the correct answer must be indicated

Why student authored ques7ons? AlternaFves

•  And a set of plausible alterna7ves

•  Of course, the correct answer must be indicated

Why student authored ques7ons? ExplanaFon

•  And an explana7on, in their own words

•  Useful for students who answer incorrectly

“You don't really understand how much or how li5le you know about a concept un9l you try to devise a good, original ques9on about it” “The aspect I found truly useful was the crea9on of ques9ons, which reinforced much of [my] understanding while also ac9vely making me clarify and solidify my thought processes (especially the explana9on parts)”

Why student authored ques7ons?

Student feedback

Badges Points Leaderboards

2) Hands-‐on session


Your turn: open a new window in your browser

Google: peerwise Start typing: Bri5sh Columbia…...

hVp://bit.ly/pwntu

Or go direct to the URL:

Registra7on

Step 1 – choose a name

Our recommendaFon: please use your first iniFal and your surname (e.g. sbates)

Step 2 – enter a password

Step 3 – enter the Course ID

11498

Step 4 – enter your Iden7fier

Any 3-‐digit number between

100 and 500

That’s it!

now log in…

Choose the “NTU2015 example” course

1) Your ques7ons: “view”

2) Create new ques7on

3) Preview

4) “Make changes” or “Save ques7on”

CreaFng a quesFon

1) Unanswered ques7ons: “view”

2) Choose ques7on

3) Select answer

Answering quesFons

3) Research highlights


Research highlights

•  Rela7onship between ac7vity and performance

•  The quality of student-‐authored ques7ons •  Influencing student behaviour with rewards

Ac7vity and learning

•  Generally, students: – Par7cipate beyond minimum requirements – Engage in community learning, correc7ng errors – Create problems, not exercises – Provide posi7ve feedback

Correla7on with end of course outcomes



Ques7on quality

Comprehensive categorisa7on of >50% of repository for two successive academic years Principal measures to define a ‘high quality ques7on’

-‐ cogni7ve level of ques7on -‐ explana7on quality -‐ other binary criteria

Category Description 6 Create (synthesise ideas) 5 Assess 4 Analyse (multi-step) 3 Apply (1-step calcs.) 2 Understand 1 Remember

Cogni7ve level of ques7on

0%

5%

10%

15%

20%25%

30%

35%

40%

45%

50%

1 2 3 4 5 6

Taxonomic Category

Perc

enta

ge o

f Sub

mitt

ed Q

uest

ions

First semester N = 350

Second semester N = 252

Results: Question level Physics 1A / 1B 2011

•  Badge / achievement system •  Randomised controlled trial (n > 1000)

Authoring ques7ons Evalua7ng ques7ons

Answering ques7ons Other

Do rewards work?

No difference between groups with respect to answer correctness

+ 13% + 22%

DistribuFon of days of acFvity differs significantly between groups: p < 0.001 DistribuFon of answers per student differs significantly between groups: p < 0.001

Implica7ons

•  Badges can have a measurable posi7ve impact on some student ac7ons

•  No nega7ve effects on ac7vity or percep7ons: a low-‐risk proposi7on

Please join us! hVp://www.peerwise-‐community.org/

These slides hVp://bit.ly/batestalks

4) Q & A


Publica7ons “Student-‐generated content: using PeerWise to enhance engagement and outcomes in introductory physics courses” (S. P. Bates, R. K. Galloway, K. L. McBride), In 2011 Physics Educa7on Research Conference (NS Rebello, PV Engelhardt, C Singh, eds.), Amer. Inst. Physics, volume 1413, 2012. (Physics Educa7on Research Conference, Omaha, NE, 2011) “Student-‐generated content: Enhancing learning through sharing mulFple-‐choice quesFons” (J. A. Hardy, S. P. Bates, M. M. Casey, K. W. Galloway, R. K. Galloway, A. E. Kay, P. Kirsop, & H. A. McQueen) Interna7onal Journal of Science Educa7on (2014) 1-‐15 “Assessing the quality of a student-‐generated quesFon repository” (S. P. Bates, R. K. Galloway, J. Riise, and D. Homer) Physical Review Special Topics:PER (2014) 10, 020105 “The Effect of Virtual Achievements on Student Engagement” (P. Denny), In Proceedings of the SIGCHI Conference on Human Factors in Compu7ng Systems (CHI '13), pp 763-‐772. ACM, 2013.

Acknowledgements These slides were based on an online workshop at the Western Conference on Science Educa7on, London Ontario in Summer 2013.

PeerWise workshop at NTU Singapore

Education