Feedback mechanisms: Efficient and effective use of technology or a waste of time and effort? Dr Robert L Allan Mr Steven Bentley School of Applied Sciences.

Feedback mechanisms: Efficient and effective use of technology or a waste of time and effort?

Dr Robert L Allan

Mr Steven BentleySchool of Applied Sciences

University of Huddersfield

Extended Degree Programme (Year 0)

• Chemistry, Biology, Maths & IT, “Physics”, H&E, Science Skills• ~120 students• Feedback mechanisms

– Grademark (Blackboard), – proforma, – group debrief,– statistical benchmarking– personal comments,

• Pressure to provide more but make it personal!• Effective?• Efficient?

Project (supported by TALI)

SPX1005 Maths & IT• ~140 students (5 tutorial groups A1, A2, B1, B2 & FCHP)• Two assignments (Formative)• Assignment 1: CV (2 page) and mini Essay (500 word)• Assignment 2: Data Analysis Report (1000 word)• Turnitin Grademark (Blackboard – known as UniLearn)• Grademark V Audacity

Grademark

Grademark

Grademark

Audacity

Attaching the feedback: Grade Center

Attaching the feedback: Grade Center

Attaching the feedback: Grade Center

Retrieving the feedback: My Grades

Retrieving the feedback: My Grades

Reflections on the implementation

• It works, but it’s not very streamlined• Blackboard/Turnitin Integration complications• A product like Wimba Voice (now part of Blackboard

Collaborate) might be more efficient.– Significant licence costs– Issue with anonymous marking still exists

• Turnitin Roadmap

Cross over design

Feedback Grademark Audacity

Assignment 1 A1, A2 & B1 B2 & FCHP

Assignment 2 B2 & FCHP A1, A2 & B1

Feedback mechanisms

Time taken to assess each assignment was recorded

Time taken to assess work

Assignment 1 Typed (min) Audio (min)

Min-Max 6-19 7.0-9.55

Mean (Median) 11.2 (11) 8.1 (8.2)

Assignment 2 Typed (min) Audio (min)

Min-Max 9.15-13.55 7.22-12.35

Mean 12.2 (12.35) 9.55 (10.0)

Questionnaire

• Easy to follow• Highlighted flaws in my overall approach to the

assignment• Identified specific errors• Indicated positive aspects to my work• Provided sufficient information on how to improve my

performance in future assignments.• I have not looked at the feedback only the mark• My overall preference is

Questionnaire contd.

Additional questions• Given your experience of the Science Skills essay on

Learning Styles please state which learning preference you have:

• Have you any suggestions about how the delivery of feedback can be improved?

Response to Questionnaire

  Typed (Grademark) Audio

Responses % Responses %

Easy to follow 58/66 88 50/66 76

Highlighted flaws in my overall approach

50/66 76 48/66 73

Indicated specific errors 55/66 83 49/66 75

Indicated positive aspects to my work

47/66 72 48/66 73

Feed forward 40/66 61 46/66 70

I have not looked at the feedback only the mark

3/66 ~5 5/66 5

Overall preference 48/66 48 25/66 38

Statistical Analysis

Critical values for 2df: *0.05=5.991, **0.01=9.210, ***0.001=13.815

Typed Feedback 

Audio Feedback

χ2 p   χ2 p  

Easy to follow 7.861 0.020 * 5.346 0.069  

Forward feedback 12.406 0.002 ** 9.624 0.008 **

Positive aspects 5.276 0.071   5.938 0.051  

Identifies flaws 9.328 0.009 ** 5.216 0.074  

Identifies errors 8.930 0.012 * 9.274 0.010 **

Ignored feedback 0.623 0.732   2.122 0.346  

Learning Preference

Numerous non-standard responses but– 62.5% of audio learners preferred audio– 25% of audio learners preferred typed– 31% of visual preferred audio

• No statistically significant relationship between learning style and feedback preference

Overview

Student preference• Typed (inserted where required)• Typed good for highlighting errors• Audio more fluid style of delivery• Both equally effective for highlighting positive aspects

and for “feed-forward”

Issues

• Anonymous marking• Compatibility: Explorer and Blackboard 9• Time consuming to upload onto Blackboard

– Size of files 1-6MB (Home Broadband)

• Student feedback– Tone of voice

• Future work– Combination Grademark and Audio– Captivate (video V audio)