Seda 2016

“You made me fail my students!”

tensions in implementing new assessment procedures

Rachel ForsythRod Cullen

Manchester Metropolitan University

[email protected]@rmforsyth

[email protected]@RodCullen

By the end of this session, delegates will be able to……evaluate the role of a model like the assessment lifecycle in

supporting educational development

Does anyone have any other intended outcomes?

8 faculties Around 1,000 courses 36,000 students

2700 on Combined HonoursLinks with over 70 professional bodies 1,500 teaching staff

plus 1,000 associates

Over 500,000 assignment submissions annually

Background

Assessm

ent and fe

edback overa

ll

5. The cr

iteria

used in

marking hav

e been clear in

advan

ce.

6. Asse

ssment a

rrangemen

ts an

d mark

ing have been

fair.

7. Feedback

on my w

ork has bee

n prompt.

8. I have

rece

ived deta

iled co

mments on m

y work.

9. Feedback

on my w

ork has help

ed me cla

rify t

hings

I did not u

nderstan

d.

33.23.43.63.8

MMU Assessment and Feedback outcomes in NSS 2008-2011

2008200920102011

NSS Questions

NSS

Sati

sfac

tion

• Interviews with key staff (purposive sample)• Focus groups with student

support teams• Grey literature review • Students’ Union termly reports• Review of external examiners’

reports• Review of academic appeals • Analysis of assignment types• Analysis of student survey

comments

Transforming Assessment and Feedback for Institutional Change (TRAFFIC) project

Examinati

on Ess

ay

Report

Portfolio

Presen

tation

Projec

t

Case St

udy

Dissert

ation

Critical

Analysis

Group Assessm

...0%

5%

10%

15%

20%

25%

MMU Most common assignment types 2013/14

level 4 - total number 1077, 65 typeslevel 5 - total number 1333, 71 typeslevel 6 - total number 1606, 74 types

Freq

uenc

y of

ass

ignm

ent t

ype

Principal findings of TRAFFIC baseline report

• Data was inconsistently recorded with respect to • assignment tasks• submission arrangements• feedback and moderation• use of grade descriptors and

assessment criteria• Assignment tasks needed

consistent support• Need to address assignment

clustering

Actual Rules• Assignments had to be aligned

to learning outcomes • Maximum of two assignments

per module (30 credits)• Grades given in percentages • All marking to be internally and

externally moderated

Principal Changes• Rewrote

institutional code of practice on assessment and associated procedures for assignment briefs, marking, moderation and feedback to students. • Embedded principle of

separation of procedures from academic decision-making about assessment

Using the lifecycle to support changeEach group has a scenario

Use the lifecycle to plan how to implement the change:

What needs to happen?Who needs to be involved?

Could you use this approach?

Feel free to write on the charts and the cards

Communications • Used evidence base to explain• Used all the channels we could think of:• Governance systems • Emails to learning and teaching leaders • Wide programme of staff development

• both course level workshops• general workshops• accredited module

• Staff newsletter• Comprehensive website

• 12000 hits on assessment section in 14/15

What’s wrong with my old criteria?

Now I have to use two sets of criteria

I still have a cognitive dissonance with the whole principle

You know, we have staff meetings and they say ‘look on CELT website’, but we wouldn’t normally in reality

You made me fail my students!

External Examiner comments 2015“Module documents, marking criteria and checklists are all now helpfully provided, and there is much more consistency in provision of guidance to the students across the programme. The rationale for mark allocations and the mark distributions appear to reflect student engagement and effort. The … use of marking criteria and the quality of feedback to students both appear to have improved. " “It is refreshing to see tutors using a marking grid to give feedback and in addition written feedback is given on the strengths and weaknesses of the work submitted. ” “ There are a number of areas which stand out as exemplars of good practice: 1) the quality of the assignment briefs and the introduction of grading criteria which conveys the expectations to students clearly and facilitates consistency in marking. 2) the quality of marking, feedback to students which is critical and constructive, second marking, and moderation; the audit trail is excellent. ”

3

3.4

3.8

4.2

MMU Assessment and Feedback outcomes in NSS 2010-2015

201020112012201320142015

NSS Questions

NSS

Sati

sfac

tion

2008 2009 2010 2011 2012 2013 2014 20153.4

3.5

3.6

3.7

3.8

3.9

4

Overall NSS satisfaction outcomes for assessment and feedback

UK averageMMU

NSS

satis

facti

on sc

ore

for A

sses

smen

t and

Fe

edba

ck

Picture credits Images from MMU, except

photo of Allsorts by clrcmck, CC licensed on Flickr Detail from "Les Lavandières de la nuit" ("The Washerwomen of the Night"), by Jean-Édouard Dargent c.1861School Door by jo_naylor, CC licensed on FlickrPostbox by electric counterpoint, CC licensed on FlickrWashing Image credit: © Copyright Neil Theasby and licensed for reuseOverloaded car by Stringer_bell, CC licensed on FlickrSisyphus by von Stuck, cc licensed on Wikimedia

Transforming Assessment and Feedback for Institutional Change (TRAFFIC)

Trying to get assessment data which…• Is accessible and helpful to personal

tutors; • Shows alignment with UK Quality Code;• Can be used to produce a Higher

Education Achievement Report (HEAR);• Identifies areas for institutional

development

Will I look forward to marking it?

Will it let students demonstrate achievement of learning

outcomes?

Will students understand what to do?

Can I link it clearly into what I’m going to teach?

Will it be straightforward to mark and moderate?

2008 2009 2010 20113.4

3.5

3.6

3.7

3.8

3.9

4

Overall NSS satisfaction outcomes for assessment and feedback

UK averageMMU

NSS

satis

facti

on sc

ore

for

Asse

ssm

ent a

nd F

eedb

ack

New Procedures • Communications • Marking• Feedback• Moderation

• Assessment Management Plans to help teams with implementation