Feedback mechanisms: Efficient and effective use of technology or a waste of time and effort? Dr Robert L Allan Mr Steven Bentley
Mar 29, 2015
Feedback mechanisms: Efficient and effective use of technology or a waste of time and effort?
Dr Robert L Allan
Mr Steven BentleySchool of Applied Sciences
University of Huddersfield
Extended Degree Programme (Year 0)
• Chemistry, Biology, Maths & IT, “Physics”, H&E, Science Skills• ~120 students• Feedback mechanisms
– Grademark (Blackboard), – proforma, – group debrief,– statistical benchmarking– personal comments,
• Pressure to provide more but make it personal!• Effective?• Efficient?
Project (supported by TALI)
SPX1005 Maths & IT• ~140 students (5 tutorial groups A1, A2, B1, B2 & FCHP)• Two assignments (Formative)• Assignment 1: CV (2 page) and mini Essay (500 word)• Assignment 2: Data Analysis Report (1000 word)• Turnitin Grademark (Blackboard – known as UniLearn)• Grademark V Audacity
Grademark
Grademark
Grademark
Audacity
Attaching the feedback: Grade Center
Attaching the feedback: Grade Center
Attaching the feedback: Grade Center
Retrieving the feedback: My Grades
Retrieving the feedback: My Grades
Reflections on the implementation
• It works, but it’s not very streamlined• Blackboard/Turnitin Integration complications• A product like Wimba Voice (now part of Blackboard
Collaborate) might be more efficient.– Significant licence costs– Issue with anonymous marking still exists
• Turnitin Roadmap
Cross over design
Feedback Grademark Audacity
Assignment 1 A1, A2 & B1 B2 & FCHP
Assignment 2 B2 & FCHP A1, A2 & B1
Feedback mechanisms
Time taken to assess each assignment was recorded
Time taken to assess work
Assignment 1 Typed (min) Audio (min)
Min-Max 6-19 7.0-9.55
Mean (Median) 11.2 (11) 8.1 (8.2)
Assignment 2 Typed (min) Audio (min)
Min-Max 9.15-13.55 7.22-12.35
Mean 12.2 (12.35) 9.55 (10.0)
Questionnaire
• Easy to follow• Highlighted flaws in my overall approach to the
assignment• Identified specific errors• Indicated positive aspects to my work• Provided sufficient information on how to improve my
performance in future assignments.• I have not looked at the feedback only the mark• My overall preference is
Questionnaire contd.
Additional questions• Given your experience of the Science Skills essay on
Learning Styles please state which learning preference you have:
• Have you any suggestions about how the delivery of feedback can be improved?
Response to Questionnaire
Typed (Grademark) Audio
Responses % Responses %
Easy to follow 58/66 88 50/66 76
Highlighted flaws in my overall approach
50/66 76 48/66 73
Indicated specific errors 55/66 83 49/66 75
Indicated positive aspects to my work
47/66 72 48/66 73
Feed forward 40/66 61 46/66 70
I have not looked at the feedback only the mark
3/66 ~5 5/66 5
Overall preference 48/66 48 25/66 38
Statistical Analysis
Critical values for 2df: *0.05=5.991, **0.01=9.210, ***0.001=13.815
Typed Feedback
Audio Feedback
χ2 p χ2 p
Easy to follow 7.861 0.020 * 5.346 0.069
Forward feedback 12.406 0.002 ** 9.624 0.008 **
Positive aspects 5.276 0.071 5.938 0.051
Identifies flaws 9.328 0.009 ** 5.216 0.074
Identifies errors 8.930 0.012 * 9.274 0.010 **
Ignored feedback 0.623 0.732 2.122 0.346
Learning Preference
Numerous non-standard responses but– 62.5% of audio learners preferred audio– 25% of audio learners preferred typed– 31% of visual preferred audio
• No statistically significant relationship between learning style and feedback preference
Overview
Student preference• Typed (inserted where required)• Typed good for highlighting errors• Audio more fluid style of delivery• Both equally effective for highlighting positive aspects
and for “feed-forward”
Issues
• Anonymous marking• Compatibility: Explorer and Blackboard 9• Time consuming to upload onto Blackboard
– Size of files 1-6MB (Home Broadband)
• Student feedback– Tone of voice
• Future work– Combination Grademark and Audio– Captivate (video V audio)