University of Surrey Learning and Teaching in HE programme Introduction to assessment Norman Jackson.

University of SurreyLearning and Teaching in HE programme

Introduction to assessment

Norman Jackson

My teaching model

Autodidactic ‘self-directed model of learning’

Didactic ‘directed model of learning’

Didactics - the art or science of teaching

My teaching model - Collaborodidactic

Our professional learning system

existing codifiedresources

teacher asfacilitator

knowledgeablecourse participants

our collective tacit knowledge gained through experience

Assessment what do you associate with the word assessment? 2 mins jot down

Assessment what is it?

In the context of a course: assessment is the means by which teachers judge what students know and can do. It involves sampling a students knowledge, understanding and capabilities, making inferences about her responses and estimating the extent to which learning is demonstrated. How? How much?

Who does it?self, peers, tutors or employers

About what?achievements,competency,potential,aptitudes, attitudes,motivations

What criteria?What is valued in grades/marks?

What types of knowledge/skill?What levels of understanding?

What formsof evidence?

* Overload of students/staff* Too many assignments with same deadline* Insufficient time for completion of assignments* Insufficient time for staff to mark* Inadequate feedback to students* Wide variation in assessment demands of different units* Wide variations in marking and use of criteria* Variations in marking within a module* Fuzzy or non-existent criteria* Undue precision and specificity of marking schemes* Students do not know what is expected of them* Students do not know what counts as good or bad

George Brown with Joanna Bull and Malcolm PendleburyAssessing student learning in higher education

What do you perceive are the weaknesses in the way we assess students’ learning in HE?

Why do we do it?Some reasons for assessing students

* to provide feedback to students to improve their learning* to motivate students* to diagnose a student’s strengths and weaknesses* to help students develop their capacity for self-assessment* to provide a profile of what a student has learnt

* to pass or fail a student* to grade or rank a student* to show a student is capable of proceeding* to provide information for future selection* a licence to practice

* to provide feedback to lecturers on students’ progress* to improve teaching * to evaluate the strengths and weaknesses of a course

Typology of assessment functions

Diagnostic - assessment at the start or during a course to provide a basis for a programme of development usually at the start of a course.

Formative - in course assessment to provide feedback on progress to aid further development

Summative - assessment about what has been learnt normally at the end of a programme.

What has been your best/worst personal experienceof being assessed?

What are the main difficulties and problems you have encountered when assessing students?

Assessment

Any procedure used to estimate a student’s learning for whatever purpose.

Some trends in assessment over last 20y

From Toend year exams end- and in-module assessment written unseen exams to coursework and examsentirely tutor led mixed tutor/student-ledimplicit criteria explicit criteriacompetition competition & collaborationproduct assessment product & process assessmentobjectives intended outcomessubject know/under other types of know/und valuedA level entry diverse entry cannot assume common background knowledge

What impact have these changes had on teaching, learning and standards? Trends in degree classifications

PropositionGenerally speaking students’ learning is driven by assessment.Students put most effort into those aspects of the course that are assessed.

What you assess and how you assess it will have a major influence on what students learn and how they learn it

So before you assess you need to decide what you wantthem to learn and why you want them to learn it.

What types of learning do you expect studentsto gain and demonstrate in your modules?

BEng/MEng COURSE* weekly lectures + problem sheets & classes + exam* as classes got bigger teachers stopped marking problem sheets, students stopped doing problems and ‘hid’ in classes.* exam marks dropped from 55-45%

PROBLEM Teaching response to doing more with less resulted in poor learning outcomes

WHAT WOULD YOU DO?

Example of useful shared experiential knowledge about assessment

WHAT THE COURSE TEAM DID

BOLD EXPERIMENT* regulation - do 50 out of 80 problem sheets or don’t sit exam* 6 ‘problem solving’ sessions run by administrators * peer assessment of problem sheets using marking scheme * no quality control!* sheets handed back immediately with peer comments* same lectures, same problems, same marking criteria, same exam, only exam marks counted

RESULT* average mark increased from 45% to 80%!

what does this tell us about the way students learn?

Giving feedback to students to help them learn

* set target for return of marked work * try to give feedback as close to the assessment as is possible* connect any grades to the commentary* if there are many comments - summarise the key points* balance positive and negative* indicate what the student needs to do to improve* make further suggestions for follow-up reading* encourage students to evaluate themselves using the same criteria* relate feedback to the assessment criteria

Feedback to students on their performancein formative assessment is an issue

why? * they may not get any* when they get it it might be too late to help them* it may not help them understand how they can improve* it may not reflect the criteria by which they are judged

Some ways of providing written feedback* Comments on assignments / exam scripts* Model answers in handout/bulletin board Feedback forms* strengths / suggestions for improvement* checklist against criteria* checklist against structure/format of assignment* self-assessment questions with answers

Use examples in Sue and Trevor Habeshaw & Graham GibbsInteresting ways to assess your students

Examples of principles for assessment1 The purposes and focus will be clear2 It is integral to course design3 What is being assessed, how judgements are made and what is expected of students will be clear4 Assessment tasks will be valid and manageable5 Assessment workloads will be realistic6 It will be free from bias and fair to all students7 Marking and grading will be consistent8 It will provide feedback to students to support learning9 It will provide feedback to teachers to inform them about the effectiveness of their teaching.10 The assessment process will be quality assured and open to independent peer scrutiny

What principles do you base your assessment on?

Perspectives on assessment

RELIABILITYConcerned with consistency / reproducabilityof measurement/marking

Potential sources of inconsistency* consistency of marking by individual teachers* consistency of marking by different teachers* test-retest reliability* student inconsistency’s ‘good/bad days’* changes in context e.g. increasing number of students

Illustrate using research studies. How might we improve consistency of marking / grading?

Perspectives on assessment

VALIDITY - a form of truth seekingis assessment fit for purpose ‘appropriateness of assessment tasks’

What assessment methods can be usedto assess what types of learning? Paired or small group discussion then build a collective map.

Place of assessment in a rational planning model for course design and delivery

1 Aimspurposes/ goals for learning

3 IntendedLearningOutcomes the learning that is likely to result from these steps

2 Objectives the steps toachieve these goalsassessment in design

4 Teaching, learningand assessment strategies to promote learning

5 Judgementson whether and how well ILOs have been achieved

intendedlearning

learningprocess

actuallearning

THAT ENCOURAGES STUDENTSTO BEHAVE IN WAYS THAT WILL ENABLE THEM TO ACHIEVE THE INTENDED OUTCOMES

ASSESSMENT ALIGNEDTO LEARNING OUTCOMES1) assessment methods that are valid to enable required learning to be demonstrated2) criteria that enable learning to be evaluated and show students what is expected

EXPLICIT INTENDED LEARNING OUTCOMES expressed in terms of knowledge, understanding, skills and perhaps values and behaviors.Nature of understanding defined by verbs

An outcomes approach to course design

teaching system what the teacher does to promote and evaluate learning

learning system what the student does to learn anddemonstrate learning

Constructive alignment Biggs (1999) encourages teachers to think about

the key to describing learning is in the use of verbs that relate to understanding in a particular context.

Biggs SOLO taxonomy - four layers of verbs 1) identify, memorise, do simple routine procedures2) list, describe, combine, perform skills in sequence3) analyse, relate, apply, argue/justify, compare/ contrast4) generalise, synthesise, theorise/ hypothesise, reflect

What verbs are you using in your assessment criteria?Are they consistent with stated learning outcomes?

Marking/grading models Royce Sadler

HD - pass with high distinction, D=pass with distinction, C=pass with credit P=pass

1Grade awarded according to mark ranges

grade mark rangeHD 85-100D 70-85C 60-75P 50-60PC 40-50 F <40

2 Grading according to qualitative descriptors

GradeHD exceptional performance complete and comprehensive understanding, mastery of skills,….D excellent performance, high level of understanding of subject, development of relevant skillsC good performance and level of understandingP satisfactory performance and an adequate understanding of basic subject matter

3Grading accordingto pre-determineddistributions

grade proportion of studentsHD 4-6%D 8-12%C 20-30%P 45-55%F 5-15%

4 Grading according to objectives

grade main minor objectives objectives achieved achievedHD all allD all mostC most mostP most someF few/none some

Use the engineering or other benchmarkstatement to illuminate this model

Assessment models!

Current undgrad. model

Normative rather thancriterion referenced.

Grading scale (100pt) culturally aligned to5 bands of honours system

Standards represent a configuration of learning characteristics in which compensation and trade-offs are a normal occurrence (Sadler 1987)

We want to view learningand performance holistically

QAA Benchmarking model

Criterion referenced

Two or three grading bands.Relationship to grading scales and honours classificationopen to interpretation

Standards represent achievement evidenced against specifiedcriteria. Outcomes must be satisfied in full. Compensationnot permitted?

Requires two stage decision making and judgements about satisfying allminimum criteria

What marking frameworks /grading models do you use?

What is the marking practice/culture in your discipline and howdoes it affect the use of thegrading scale?

Illustrate different grading profilesin different subjects

How does assessment in your module relate tothe programme as a whole? Connect to programme specification and curriculum maps

12345678etc

curriculum building blocks

TPA TPA TPA

TPA TPA TPATPA

TPA TPA

TPATPA TPATPA

TPA TPA TPA

TPA

TPA

TPAPAPA

P

P

P

P P

P

A B C D E F G Hprogramme outcomes

T=taught P= developed through practise A=assessed

Benchmarks

12345678etc

Practical Assessment Design Learning Activity

Step 1 30mins gps of 3 using example provided1 Design and justify an assessment strategy that you feel would be valid 2 Develop examples of criteria that you would use to evaluate the achievement of learning outcomes3 Identify the principles that you have used in this exercise4 Identify any issues that the process raisesProduce a flip chart posterStep 2 5 develop criteria for self evaluation of poster and evaluate/grade your own6 evaluate the other posters using the criteria and put grade on yellow post-it 7 group discussion - what did we learn?

Useful resources for assessment

George Brown with Joanna Bull and Malcolm PendleburyAssessing student learning in higher education, Routledge.

Sue and Trevor Habeshaw & Graham GibbsInteresting ways to assess your students

Dai Hounsell, Mary McColloch and mary ScottThe ASSHE Inventory Changing Assessment Practices in ScottishHigher Education on-line at www.ltsn.ac.uk/genericcentreMany examples organised by subject/theme.

Learning and Teaching Support Network Subject Centre web siteshttp://www.ltsn.ac/subject

Learning and Teaching Support Network Generic Centre web sitehttp://www.ltsn.ac/genericcentre (Project - Assessment)