Pg cert lthe assessment 2011 slideshare version

Assessing for LearningAssessment and Feedback Methods

Phil Carey Faculty of HealthClare Milsom Academic Enhancement Unit

Outcomes

1. Recognise the importance of assessment to the learning process2. Identify the key components and principles of the assessment

process3. Reflect on and critically evaluate a range of assessment and

feedback methods

Schedule0930 – 0950 Assessment in context of constructive alignment0950 – 1030 Activity: alignment of outcomes, teaching methods

and assessment tasks.1030 – 1045 Break1045 – 1115 Introduction to e-resources1115 – 1230 Feedback (with short break)

Why is assessment so important?

1. Assessment has a major influence on:what students learnhow we teachhow students organise their studieshow individuals are able to progress

2. Staff and students spend more time on assessment than on anything else

“Assessment IS the curriculum as far as many students are concerned – they’ll learn what they think they’ll be assessed on, not what’s in the curriculum or what’s been covered in class” Ramsden (1992)

Course content Assessment

Teaching methods Study approach

Assessment

Surface Deep Strategic

Operational Surface Tactical

Teaching teaching and understanding understanding

19-minute award-winning short-film about teaching at University: ((( http://www.daimi.au.dk/~brabrand/short-film/ )))

Students take responsibility for their learning. ‘Deep’ rather than ‘surface approach’

Course content Assessment

Teaching methods Study approach

Assessment

Teaching and learning activities

Designed to meet learning

outcomes

Intended learning

outcomes

Assessment methods

Designed to assess

learning outcomes

How will they learn? What do we want our students to know?

How will we know the students have learnt?

Constructive alignment (Biggs 1999)Learner constructs their own learning through learning activities

‘High levels of detail of learning outcomes and assessment criteria, allow students to identify what they ought to pay attention to, but also what they can safely ignore’. (Gibbs 2010: 25)

Structure of the Observed Learning Outcomes (Biggs and Collis 1982)

SOLO taxonomy

Bloom’s revised taxonomy (Anderson and Krathwohl 2001)

Level Bloom Solo Verb

6/7 CreatingExtended abstract Design, hypothesise, construct, reflect

6/7 Evaluating

5/6 AnalysingRelational Apply, compare, analyse, debate

5/6 Applying

3/4 Understanding Multistructural Classify, illustrate, report,

3/4 Remembering Unistructural Identify, name, define, label, write

All learning outcomes include:

Verb, content, context

What’s wrong here?

1. Identify the key constitutional structures of the European Union and critically analyse the origins and effects of these structures by using the conceptual tools of comparative politics.

2. Recall the fundamental structural, mechanical and electrical engineering.

3. Write a report on the decontamination of reclaimed land.

4. Explore global provision of emergency healthcare 5. State the six categories of Bloom’s taxonomy.

Feedback conceptYear 1: confidence Year 2: improvement‘when I had done something well I got great feedback and this boosted my confidence’

‘To enable growth and learning feedback should be timely but it is not’

‘.....happy to help and give feedback on my work.... Tutors are encouraging’

‘The worst experience at this university is with feedback. They might tell us what we did wrong…, but not telling me how to improve it does not help at all.’

Comparisons of main concepts and associated sentiments and strength

Concept Year 1 Likelihood % Year 2 Likelihood %

Course Favourable 33 Favourable 38

Feedback Favourable 10 Unfavourable 21

Coursework Favourable 06 Unfavourable 04

‘Mirror’ student year 1 and year 2 surveys

Student views of feedback

Questions 2010 NSS 2011 NSS

satisfied satisfied

1-4 The teaching on my course 83% 84%

5-9 Assessment & feedback* 66% 68%

10-12 Academic support 75% 77%

13-15 Organisation & management 73% 75%

16-18 Learning resources 80% 80%

29-21 Personal development 79% 80%

22 Overall satisfaction 82% 83%

National Student Survey results for the UK

* Universities across the sector underperforms in assessment & feedback

Institutional survey on student views of feedback(Carey, Milsom, Brooman & Jubb, 2010)

Method: anonymous questionnaire

Focus of questionnaire: Student experience of assessment feedbackStudent expectations of assessment feedbackUtility/value of different feedback types

Questionnaire designed and tested in February 2008

Disseminated to undergraduate students in four Faculties in March 2008:Business & LawMedia, Arts & Social ScienceScienceHealth & Applied Social Sciences (only those undertaking a professional qualification)

Distribution (n=1409)

• Nearly half (46%) waited for more than 4 weeks for feedback• 85% got feedback at the same time or after they were given marks

How quickly do students expect their feedback?

Science students want their feedback more quickly

(p < 0.001)

Student views on the standard of feedback they receive

Marking criteria • Informed about criteria - 80% agreement • Understood criteria - 72% agreement

Standards and fairness• Confident of fairness - 76% agreement • Marking is accurate - 68% agreement

Effect on learning • Clarified understanding - 59% agreement• Helped future performance - 57% agreement

Ranked usefulness of feedback methods

1. One-to-one discussion with module staff2. One-to-one discussion with personal tutor3. Written comments on feedback sheet4. Annotations on scripts5. Individual email with specific comments*6. General feedback to whole class7. Feedback in work placements*8. Group email with general comments*9. Tick-box assessment sheets10.Feedback from other students*

* Low reported experience of this method

The influence of faculty

Science students rated the following less useful than others did: • One-to-one discussion with module staff (p<0.000)• One-to-one discussion with personal tutor (p<0.000)

Health Students rated the following more useful than others did:• Individual email with specific comment (p<0.000)• Feedback in work placements (p<0.000)• Tick-box assessment sheets (p<0.000)• Feedback from other students (p<0.000)

The impact of the ‘engaged student’

• Analysis assumes collection of work as an indicator of engagement

• 42% strongly agreed with the statement “I collect marked work as soon as it is available”

Association between this and:–understanding criteria (p<0.001)–being informed (p<0.05)–believing marks reflects standards (p<0.001)

What do students tell us that they want?

Overall students appear fairly satisfied, but…Want more opportunities for one-to-one feedbackPrefer easy to access modes of feedback

Students are somewhat conservative – less interest in peer assessment or group feedback

a result of limited experience?less interest in feedback related to work-based learning

implications for graduate employability

REMEMBERAssessment for students is a personal event, but……staff view assessment in terms of groups of students

What should we do?

1. Manage expectations2. Enhance student engagement with the assessment &

feedback process3. Reinforce notions of student’s own responsibility 4. Consider disciplinary difference in assessment &

feedback mechanisms.5. Resist excessive centralisation of feedback mechanisms 6. Avoid innovation for the sake of innovation7. Recognise the importance of training and experience for

mastery and confidence in assessment methods

NEXT WEEK• Prepare for discussion of assessment method

rationale with peer learning groups• Identify assessment method

– Consider alignment with learning outcomes – Think about the students’ skills level for this

technique– Explore issues of equality and diversity– Look at compliance with university policy– Identify appropriate feedback mechanisms