Mechanised Assessments that Promote Deep Learning …and prevent plagiarism!! Gareth Denyer & Dale Hancock, University of Sydney
Feb 12, 2016
Mechanised Assessments that Promote Deep Learning
…and prevent plagiarism!!
Gareth Denyer & Dale Hancock,University of Sydney
Multiple Choice• Rapid, convenient
– Excellent for large classes• BUT…. do they encourage surface learning?
– Questions often designed to test unambiguous facts• Student PERCEPTION
– adopt a ‘rote’ approach– Names, facts, numbers
• Attempts at deeper, conceptual based possible…– BUT… often lead to ambiguous questions– At worst, questions favour less able students because better
students confused!• VERY SKILFUL JOB!!!
Marking MCQs• No credit for near misses or process
– No point in leaving answers ‘blank’• So you never know ‘why’
• Guessing can give 20%– Negative marking?
• Intimidates students• Forces meticulous non-ambiguity• Further driving surface learning!
• Post-Exam revision of mark scheme difficult– Unlike SAQs
• where mark scheme can be dynamically changed– Questions often ‘discarded’
• Disadvantages good students
Graded AlternativeNB. This is a SURFACE example!!
What is the capital of Australia?A MelbourneB SydneyC GlasgowD CanberraE Auckland
– The more complex the question, the more important partial marks become.
1
0.5
0.5
0
-1
XL Based Graded SolutionTypical student answer grid
Key Sheet
Marks
Calculations
• Blah, blah… What is the approximate ε450nm for the coloured compound formed in the assay?
A.0.199 mM-1 cm-1
B.250 mM-1 cm-1
C.4 mM-1 cm-1
D.2 mM-1 cm-1
E.19.6 mM-1 cm-1
Each option reveals a particular mistake
Especially good with a multi-step calculation
Can reward process
Opens up Deeper Styles
• The “What if?” Cause → Effect– What would be the consequences of inhibition
of lipolysis during the first few days of starvation?
• Shades of Grey– You decide that you need 50 mU of citrate
synthase (CS) in the cuvette. What is the MOST PRACTICAL way of doing this?
Advantages• Makes marking flexible
– Like an SAQ!– Post-hoc changes possible– Provides an alternative way of scaling
• Staff less stressed– More extrapolative questions
• Less attention to rigorous ‘fact’• Less concern about ‘getting it perfect’
– No conflict with colleagues over ambiguous questions• Feedback better
– Each option “useful”
Plagiarism!
• Much attention given to detecting cheating in assignments– Puts you into a conflict situation– But exams make up most of the marks!
• Cheating is easy in MCQ exams– Hand signals– Pattern recognition
• Several cases – A healthy literature!
1 2 3 4 5 6 7 8 9 101
4
7
10
0
5
10
15
20
Similarity of Incorrect Answers
Column
Row
adapted from data in: Harpp & Hogan (1993) Crime in the classroom: Detection and prevention of cheating on multiple-choice exams. J. Chem. Educ. 70, 306
Islands of corruption
Wise Words
Universities obviously want to stamp out cheating
But they also do not want any students to be falsely accused
Statistical evidence is generally not enough.
Pursuing a case is traumatic for all those involvedEspecially if an unwilling cheatee!
1 in 653,000
The Versions SolutionCreate FOUR versions of the same paper
Questions in same order, options just rotated
Layout so that each version is isolated from its clones
Very easy in Word
A database solution can generate even more versions.
Implementation
• Sweet Justice!!– Appropriate recompense without confrontation– Lots of ‘evidence’
• Sometimes even ‘check’ questions copied
• Easy– Just Word and Excel– Rotations take about 10 min per 100 MCQs
Take Home Messages
• Graded MCQs give flexibility• Different versions disincentivise cheating• All easily implemented
– Word and Excel• We are happy to provide templates and
instruction
Have a go! Easy and Liberating!