Lets Abolish Feedback Brian Whalley (Lately of Queens University of Belfast) and put something better in its place.

Let’s Abolish ‘Feedback’

Brian Whalley(Lately of Queen’s University of Belfast)

and put something better in its placeand put something better in its place

we’ll be

• Looking at existing ‘feedback’

• Looking at what feedback is, variously

• Looking at some simple educational systems

• Marks, remarks and students’ needs

• Making suggestions for improvement

The NSS and the feedback issue

• What do the data really mean?

• An example of ‘institutional panic’

• I don’t think they mean a great deal!

• even if they dodo then how can they be improved – given that

• feedback is the best way to improve performance (see Nichol, Gibbs, Race, and especially Geoff Petty (Evidence-based Teaching)

What is ‘feedback’

• It depends upon whom you ask!

• Plenty of good statements so well look at a few consequences

• Nicol and Mcfarlane Dick

• Alastair Irons

• (‘Enhancing learning through formative assessment and feedback’ 2008)

The Ten Principles of Assessment and Feedback• 1. Help clarify what good performance is

• via aims, criteria, standards

• 2. Encourage ‘time and effort’ on challenging learning tasks.

• 3. Deliver high quality feedback information that helps learners self-correct.

• 4. Encourage positive motivational beliefs and self-esteem.

• 5. Encourage interaction and dialogue around learning (peer and teacher student)

(Nicol 2007)

10 principles continued

• 6. Facilitate the development of self-assessment and reflection in learning.

• 7. Give learners choice in assessment – content and processes.

• 8. Involve students in decision-making about assessment policy and practice.

• 9. Support the development of learning communities.

• 10. Help teachers adapt teaching to student needs.

How do we use exams (and coursework)

to deliver these?

Nine ‘disadvantages’(From Irons, p 25-6)

• Students don’t make use of feedback

• Doesn’t actually contribute to learning

• Only there to justify the mark given

• May be categorial in tone and not explicit

• No opportunity to enter into dialogue

• Emphasises power relationships

• The potential to include bias

• May actually foster rote learning

• Might be inappropriate

To some, possibly large, degree these are because of differences in what is meant by ‘feedback’.

If feedback is so important then why is it referenced so infrequently? Where do you

find a definition?

•Well,

•we all know what feedback is, don’t we?

•Don’t we?

‘your’ module (swap)

(5 minutes)

• Draw a time line (start of module to end)Draw a time line (start of module to end)

• Tick for start (when do you start?)Tick for start (when do you start?)

• Tick for end (what denoted the end?)Tick for end (what denoted the end?)

• Note the hours to be spent (lectures etc)Note the hours to be spent (lectures etc)

• Put a symbol to mark any Put a symbol to mark any AssessmentAssessment event event

• Put a symbol to show when Put a symbol to show when ‘feedback’‘feedback’ is given is given

• Note below the Note below the typetype of feedback given (however you of feedback given (however you define it)define it)

A(x)

F(x)

Reflect, are you happy with this?Reflect, are you happy with this?

Discuss your chart with a neighbourDiscuss your chart with a neighbour

Are you happy with each others, and indeed, the feedback process?Are you happy with each others, and indeed, the feedback process?

when to give feedback

• When can you?

• When should you?

• When do you?

• Do others in your department agree on when you do?

• Do others in your department give ‘proper’ feedback anyway (module scheme)?

A definition

Feedback and Feedforward in a control system sense

Do ‘exams’ provide a reference input (or even output?)A MCQ might do this. Criterion Referencing might too.

Educational ‘ramp up’•To raise the level of ‘attainment’

•We need controlled input, ramping up.

•Output, achievement, results

attainment

Ee Marks, no remarks forExam events

Ese Ee Delayed marks and remarks forEssay event Ese

TTask set with CriterionReferenced assessment

Semester start Semester end

Ee endstart

‘Attainment’ Ee

Delayed marks and remarks for Exam event, EeEe

Can I be more constructive?

AT

MR AT

MRTask Activity Marks Remarks

AT

MR REe

MAT

MR

Gradient = ‘learning rate’

Two Activity c/work

Two Activity c/work + exam

How do we increase the gradient?

Allison Rossett(First Things First)

Performance analysis Training needs assessment

Current Optimal

Key Areas of Need

RecommendedSolution System

PerformanceSupport

The seductive nature of the new media is another good reason to add performance analysis right up front. Performance analysis keeps the focus on the customers and their purposes.

The goal is to find the ‘right’ bundle of interventions, not to take a spin with podcasts or Second Life.

Training needs assessment

New Courses

Solution Partners

Revised Courses

Job Aids

E-Learning

‘Everyone’ needs trainers

Umm, sometimes it works ……

Teamwork

Feed forward

It is not judgemental. The negative connotations of past failures are banished There is no such thing as failure just Feedforward. It is much easier to deliver. People are less defensive when discussing future performanceFeedforward is taken less personally, provokes less resistance.

Marshall Goldsmith

Itiel Dror

• Control – handing appropriate responsibilities to students

• Challenge – student abilities

• Commitment – student commitment to learning opportunities

What might be the best ways (note plural) todevelop these?

Using cognitive psychology

6 Competenciesstudents need to gain

•Competence – encouragement by challenge and remarks to achieve skills levels

•Confidence – promoting remarks to show themselves, and others, their achievements

•Critical thinking – which is what we have been wanting all along in 'Thinking skills’, used in problem solving

•Creativity – in what students do and how they do it

•Collaboration – bringing in team-working and ethics

•Commonality – of purpose, to achieve specified (and unspecified) objectives

•Curtiosity – which is more than curiosity.

Marcia Mentkowski

Mihály Csíkszentmihályi

Assessment for Learning(Geoff Petty - Evidence-Based Teaching)

‘Seven principles common to high-quality learning and achievement’ (p164)

1. Students must see the value of the learning

2. Students must believe they can do it

3. Challenging goals

4. Feedback and dialogue on progress to the goal

5. Establishing the structure of information and so its meaning

6. Time and repetition

7. Teach skills as well as content

Feedback and performance

• This is not as difficult as we might think (ahem)

• Changing the way we work at things

• Using informal learning and performance ideas

Problemspecified

Manipulation of

system

Problemsolution

CluesGuidance

Help

Provision ofFeedback on

solutions

RulesSystem

ExamplesProcedures

Task Task Task

ResourcesSupport Support

Assessment

T Tacit knowledge required

S Sticking point(s) likely

TS

Fig 5.1 A temporal sequence describing a rule-based learning design, In Oliver et al. 2007. Describing ICT-based learning designs that promote quality learning outcomes

R R

Ron Oliver et al. Schema, also

Communities of learning• We need to develop these:

• At a variety of scales of operation -

• In particular, at our own institutions -

• Student-centred but bringing in:

• Tutors, pedagogy*, employers, internet,

• Web 2.0, digital repositories,etc

• Wisdom of Crowds (Surowiecki, Shirkey) etc

•* Using cognitive psychology

Collective Learning: Consuming knowledge Connecting knowledge Contributing knowledge Charting knowledge

Assessment

• Alignment?

• To fit with diversity of skills as well as instruction methods etc

• E-portfolios

• Transcripts and record of achievements

• Leading back to employability

Reasons to be cheerful?

• ‘If teaching was as simple as telling we’d all be a lot smarter than we are.’ (Mark Twain)

• ‘In Carol Dweck’s terms, it takes an incrementalist approach to learning which emphasises effort and improving competence. This contrasts with an entity approach, which attributes learning to ability and is focused on proving competence through grades and comparisons.

• Gordon Stewart (2008)

Towards a proper educational feedback system

What do I mean by this?

• Forget about Kolb cycles and Cowan loops* or, perhaps better, use them to give a system that works for all students (and staff)

•Construct (with ‘Alignment’*) a system (for a module) that maximises learning and understanding

(rather than memory recall)

•Use Thinking Styles (Sternberg) rather than Learning Styles

•To produce ‘proper’ alignment we really need more specific instructions per task/activity

•And to provide remarks with the marks is use a feedback in an educational control system

Delays and Troublesome Knowledge & Sticking Points

• Troublesome knowledge (Land)

• Tacit knowledge (Polanyi)

• Delays doing activities Sticking Points

• Preflights (Just-in-Time Teaching; G Novak)

Feed-forward (controlled use of advance info)

• After Action Reviews

• +ve Feedback or remarks

Things we can do, 1, general

• Recognise the needs of students and their antecedent conditions

• Too much ‘winging it’ on what they (think they) know how and what to do.

• Recognise the tutors assessment nd feedback provision is by no means ‘tried and trusted’ (was it ever satisfactory, let alone good)

• Recognise that timing is important

‘Preflights’(Warm-ups)

• Tasks set before the ‘event’ for students to do

• Could be assessed or not (gratuitous marks)

• Confidence building (in you, for themselves)

• Tackle ‘Troublesome knowledge’ beforehand

• Gets them to start something required later

• helps to avoid last-minute syndrome

• Can reflect on the task and achievement

• Make them easy to assess (if required)

• Follow up with After-Action Review (reflection)

• Link in to Criteria-Referenced Assessment

PFL examples

• Look up definitions (e.g. on Wikipedia)

• Prepare a bit of a spreadsheet formulation

• Write a bit of HTML script

• Prepare a PDF of a document

• Prepare the basis for a digital poster

• Analyse a photograph (prepare metadata)

• Extract some critical data from a table

• Prepare a Concept Map of a problem

• Find an Open Resouce app for a task

Criteria Referencing

• Shows students what is wanted in advance

• Shows the mark scheme related to requirements

• Tutors Marks and Remarks (together or not) provide External feedback for the system

• Easy to get them to reflect (ie they provide the feedback) Internal Feedback for the system

• Easier to plan what is required from the assessment

• Important for cross-checking (eg demonstrators)

• Easier to show external references (Externals)

• Pretty much what is meant by ‘feed-forward’ (ie showing the limits for energy input into the system)

What drives learning

•Not the assessment (itself)

•This (best) drives checking (controller in the system). It’s an indicator (control signal)

•Driving learning (+ve feedback)

•Internal (student doing the activity)

•External (comments from others, peers, tutor, manuals)

•Inhibitors to learning (-ve feedback)

•No remarks (‘feedback’)

•Exams with no remarks (‘feedback’) before the end of the module

students (tend to)revert to (school) type

• expectations of small classes with

• teachers they know well and who

• can give pretty immeiate remarks and even

• return work for ‘repairs’ and re-submission

• may not be using good cognitive techniques which often rely on or use

• braindumping techqnuies fro assessment

Assessment for Learning(Schools are using this!)

• making exams less dominant

• and why should they be anyway?

• what do marks and remarks mean anyway?

• having ‘care of customer’ for students, especially in year 1 (and Semester 1)

• Use class tests (MCQ, Certainty-based)

• Formal exams before the end of the semester (and proper remarks with the marks to the student -not just the External)

‘Care for customer’

• Look after Level 1

• Make the assessment explicit

• Criteria Referenced

• Match marks to criteria (explicitly)

• Give students practice in assessment (types)

If we believe in these principles of ‘deep learning’then (in general):

•Why do we set essay exams?

•and seem to rely on closed exams?

•and not used open book or seen?

•Why do we set exams at the end of modules?

•Why do we not give ‘feedback’ on exams

•Why do we believe MCQs are ‘ok’; ‘better’?

•Why there are several cetls involved with active learning etc but few with ‘Assessment’?

•……… and you can probably think of more

some references and websites

• Irons, A. 2008. Enhancing learning through formative Assessment and Feedback. Routledge.

• Rossett, A. First Things First (A handbook for performance analysis) Pfeiffer 2009.

• Siemens, G. Knowing Knowledge

• Sternberg, R. J. (1997). Thinking styles. Cambridge University Press.

• Stewart, G, Testing Times 2008 Routledge,

• Stobart, G. (2008). Testing times: the uses and abuses of assessment. London: Routledge.

• Sweller, J. (1994). Cognitive Load Theory, learning difficulty, and instructional design. Learning and Instruction 4: 295–312.