Making the Grade Moving to a GPA-based System in a UK University Dr. Gavin Brown Senior Lecturer in Biochemistry, [email protected].

Making the GradeMoving to a GPA-based System in a UK University

Dr. Gavin BrownSenior Lecturer in Biochemistry, [email protected]

A plan for the day

• Presentation – what has Lancaster done and why?

• Discussion – a role for GPA at Leeds?

• Break

• Presentation – impact on classification, effecting

change

• Discussion – how might change happen at Leeds?

• Final questions

Please ask questions as we go along!

The Lancaster GPA System

• Simplification• Comments from external examiners

- Limited use of the full marking range- Complexity of our regulations- Some inconsistency wrt operation of exam boards

• Concerns from LU staff• Burgess Review

- Ensure regulations are fair, transparent and fit for purpose - Possible future changes in the sector

Drivers for change

• An example from Lancaster’s old classification system• The mark profile consisted of an overall average plus the mark

distribution of 16 module marks (15 credits each)• To get a first you need:

- An average of 70%+ OR- An average of 68%+ and 8/16 marks at 70%+ OR- An average of 68%+ and 6/16 marks at 70%+ OR- An average of 68%+ from the best 14 marks and 8/16 at

70%+ OR- An average of 68%+from the best 15 marks and 7/16 at

70%+ OR- Exit velocity …

Simplification

• Common feedback about using the full range of marks in some subjects, but …

“I read a project where the supervisor had commented that it was the best dissertation they had read in over 20 years and was of

publishable standard. The supervisor had given it 72%. I thought that was rather unfair, I would have given it 78%”

External examiner in a humanities subject, 2006

External examiners

• Complexity• Treatment of mitigating circumstances• Inconsistency between subjects (joint degrees)• Special rules• Exit velocity

Concerns from LU staff

• Reluctance to use the full range of marks in some subjects

• Quantitative subjects wanted overall average, qualitative subjects wanted mark distribution

• Creeping complexity of the regulations• Generous condonation rules• Exams boards were fair but there were different

approaches• Inconsistent treatment of mitigating circumstances• Lots of ‘special rules’ / system complexity

Summary of ‘problems’?

• Leaving as was but introducing differential weighting• Allowing departmental/faculty variation (use of mean or

distribution)• Moving to a step marking scale based on percentages• Moving purely to a grading scale• Mapping percentage ranges to grade points• Optimal maximum GPA• Number of points on any scale / in any class• Linearity of scale• …

Marking - what we considered

Feb 2007 - Project started

May 2010 - Approval of principles

Nov 2010 - Final approval from Senate

Oct 2011 - Introduced for 1st & 2nd years

Jun 2013 - First graduating cohort

Timeline

• Qualitative work marked in grades• Quantitative work marked in percentages• Marks converted to a GP (aggregation score)• Classification based on overall GPA• Criteria for borderlines

• Resits mandatory• Minimum condonation threshold• Set aside rules for mitigating circumstances

Lancaster’s new system

Grading criteria (qualitative work)

• Must not mark in % then ‘convert’ to grade• Use objective criteria to assign a grade• Initially assign the middle grade (A, B, C, D) then

revise up or down using secondary descriptors• Use objective criteria for feedback• Bespoke marking criteria for the subject, level,

assessment type, etc.

Grading: the pedagogical change

• Marking as usual• Percentages converted to GP

0% = 0.000, 1% = 0.225, 2% = 0.450, 3% = 0.675,

…

… 40% = 9.000 …

… 98% = 23.700, 99% = 23.850, 100% = 24.000

Marking in %s (quantitative work)

• 24 point scale• A+ and 100% both equal 24

Grade Point Average

GRADE(qualitative work)

PERCENTAGE(quantitative work)

AGGREGATIONSCORE 0-24

(a ‘grade point’)

OVERALLAGGREGATION

SCORE(a GPA)

CLASSIFICATION

Calculation of module result

Calculation of final result

The Interactive TranscriptA big win!!

Classification

• If a student’s overall GPA falls into a borderline then the higher award is given if:- Half or more of the credits are in the higher class or- The final year average is in the higher class

(exit velocity)

• Exam boards can make individual cases to Senate

Borderlines

• Grading scheme linked to objective marking criteria- Promotes use of the full marking range- More straightforward marking- Clearer opportunities for feedback

• Percentages still used for quantitative assessments

• Classification based on mean only

• Less condonation and only if a satisfactory attempt has been made

• Exam boards given greater guidance (and less discretion)

How is the new system better?

• I am very supportive of the shift away from percentages to A,B,C grading• I felt that the new marking system is clear and equitable in ways that our own

new 17-point system is not • I am happy to note that Lancaster have moved to a conceptual scale of

marking (e.g. A+, A, A- etc). A percentage scale, particularly in Arts subjects, discourages marking at the top end, thus this new procedure will hopefully encourage assignment of marks across the entire scale

• I very much welcome the aggregation initiative with the use of letter grades, which will certainly solve the age-old problem of borderline marking and not using the full range of grades, particularly at the upper end of the scale

• Uniquely both new and old systems were in operation this year and I can’t help thinking the former will represent an improvement. It will be interesting to see whether the A band gets used more adventurously than the old first class band was, which is surely what it’s designed to do. Greater use of the A band breadth may also have the effect of pulling up B/C/D marks ie. because the dizzying heights of an A+ are a possibility now, that puts lower grades in a revised perspective

External examiner feedback – positive

• I wonder however if the new assessment system being phased in will be as easy to work with and as transparent to potential employers etc.

• I found the new marking scheme to be very confusing (as I was looking at 2nd and 3rd year work there were letter grades, marks out of 24 and %s used). I would recommend the University to use either letter grades or %s

• The “24 point” marking scheme is completely new to me, and is not something I have encountered any other university where I have been external examiner, nor in my own institution. I have not yet seen any meaningful justification why Lancaster University should employ this system, or how it helps students or examiners

• I am aware that the new assessment regulations are in operation and I broadly welcome its introduction. Despite, its introduction I would still encourage the staff in the department to utilise the full range of marks as my sense of conservatism in marking at the top end still has potential to create problems regarding a student’s classification as the letter grade mark needs to be translated into a numeric mark

External examiner feedback – negative

A Role for GPA at Leeds?

DiscussionPossible questions to consider …

• Are there any ‘problems’ with the current Leeds system?- If yes, how might a GPA system help?- If no, could enhancements be made?

• At what point would GPA be used (component, module, overall)?

• Would you want a mixed system (%s and grades)?• How much subject variation would you want/need?• What’s in it for your staff?• What’s in it for your students?• What might the reception be to change and why?

Lancaster GPA – Impact on Classification

• Not easy! Changing more than one variable

• All graduating cohorts from previous 3 years

• ‘Qualitative subjects’ – percentage ranges

mapped to grades

• ‘Quantitative subjects’ – percentages converted

to scores

Minimising risk: data modelling

Distribution of first class averages for final year

% o

f firs

ts

Aggregation Score

17.5-17.9 18.0-18.9 19.0-19.9 20.0-20.9 21.0-21.9 22.0-22.9 23.0-24.00%

5%

10%

15%

20%

25%

30%

35%

40%

45%

2010/112011/122012/13

Classification results – overall

2010 2011 2012 2013 20140%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PassThirdLower secondUpper secondFirst

Year

% o

f coh

ort

Classification results – science and technology

2010 2011 2012 2013 20140%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PassThirdLower secondUpper secondFirst

Year

% o

f coh

ort

Classification results – business and management

2010 2011 2012 2013 20140%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PassThirdLower secondUpper secondFirst

Year

% o

f coh

ort

Classification results – arts, humanities and social sciences

2010 2011 2012 2013 20140%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

PassThirdLower secondUpper secondFirst

Year

% o

f coh

ort

• Several factors at work (increasing entry grades,

compulsory resits, final year resits, etc)

• Increase in overall proportion of ‘good degrees’

(GDs) broadly in line with increased intake

qualifications

• Quantitative subjects show more GDs but with

variability in firsts

• Qualitative subjects show slight improvement in

GDs but number of firsts improved

Classification conclusions

Lancaster GPA – Effecting Change

• Took 3.5 years• Started with a cross-faculty working party (including

‘difficult’ subjects)• Consultation with other universities• Establish principles• Outline planning permission from Senate• Several rounds of consultation with amendments and

compromises• Visits to departments• Post-implementation monitoring group

How did we go about making a change?

• Departmental staff meetings• Faculty teaching committees• Students’ Union Council• Student Course Representatives• Staff and Student FAQs• Flyers + emails• Student Newspaper• Students’ Union social media

Communication with staff and students

• How will work be marked in the new scheme?• What do I put on student work?• How are final marks calculated?• What happens if a student fails my module?• What do I do if a student has mitigating circumstances?• What if a student has still failed after resitting?• What does a student need to progress between years?• What are the penalties for late submission?• What will appear on a student’s transcript?• …

Staff FAQ

• Tackle the cause, not the symptom• Be prepared to compromise• Use enthusiasts• The lead must invest a lot of time• Consult with external examiners and employers• Introduce parallel benefits for students• Don’t underestimate system implications (student

records systems, Turnitin, etc)• Piecemeal is problematic!

Lessons learned

DiscussionPossible questions to consider …

• What would be the process of change at Leeds?• Who would lead it?• Who needs to be involved?• What role would/might students have?• Where would the hurdles be?• Can you get the data for modelling?• Is it worth it?

Any final questions …?