Stephen Russell 2008

1

Assessment in Australian universities:

What they say they do to engage students

Stephen RussellDefence And Systems Institute, UniSA

Objectives

To determine:

1. The principles of best practice in using assessment to engage students recommended by Australian universities

2. The degree of support by the universities for each principle

ATN Assessment 2008 Adelaide

2

Why?

Marton, Säljö, Biggs, Entwistle & Ramsden started questioning good learning and assessment in 70s and 80s

We KNOW about good assessment practice But knowing is different from practicing good

assessment practice Race (2003) asserts that assessment

practice in the UK is ‘broken’ Angelo (1996) asserts serious deficiencies in

US assessment practiceATN Assessment 2008

Adelaide3

Initial Assumptions

Australian universities are aware of and concerned about good assessment practices

Australian universities publicise their particular view of the principles of good assessment practices

These principles are available on the Internet University web-sites are user friendly

ATN Assessment 2008 Adelaide

4

Methodology

An initial set of principles was derived from the literature: Assessment should measure learning outcomes Criteria should be clear, explicit, consistent,

justifiable, evaluated and revised Assessment should help students to learn Feedback should be explicit, objective, and timely Workloads should be appropriate A variety of assessment tools should be used Assessment purposes should be clearly explained

ATN Assessment 2008 Adelaide

5

University views of good assessment practice were derived from their websites (Australian Education Network, 2008)

Minimum of 5 principles Policy statements were not included 28 out 39 university guidelines included Policies translated in systems engineering

requirements (-like) statements – concise, simple, specific, ‘should’ imperitive

ATN Assessment 2008 Adelaide

6

Methodology

No. Principle

1. Objectives

Rqt.1.1 Assessment should help students learn

Rqt.1.2 Assessment should be criterion-referenced

Rqt.1.3 Assessment should assist students achieve professional standards of work

Rqt.1.4Assessment should judge achievement of learning outcomes

Rqt.1.5Assessment should help students to improve generic (graduate qualities)

ATN Assessment 2008 Adelaide

7

50%

Results

2. How students learn

Rqt.2.1Assessment should be progressive over time

Rqt.2.2A diverse range of assessment methods should be used

Rqt.2.3Assessment tasks should be appropriate, fair and equitable

Rqt.2.4Students should be involved in their own assessment

ATN Assessment 2008 Adelaide

8

50%

Results

3. Assessment as part of learning

Rqt.3.1 Assessment should be an integral to course design

Rqt.3.2Assessment tasks should allow demonstration of deep learning

4. Assessment Tasks

Rqt.4.1 Assessments tasks should be clear

Rqt.4.2 Assessments tasks should be specific

Rqt.4.3 Assessment tasks should be meaningful (real life)

Rqt.4.4Assessment tasks should address a variety of learning outcomes

Rqt.4.5 Assessment tasks should minimise chances of plagiarism

ATN Assessment 2008 Adelaide

9

50%

Results

5. Assessment Criteria

Rqt.5.1 Assessment criteria should be clear

Rqt.5.2Assessment criteria should be specific (describe required standards of – knowledge, skills, competencies and capabilities)

Rqt.5.3Assessment criteria should be valid (measure what is intended)

Rqt.5.4 Assessment criteria should be justifiable

ATN Assessment 2008 Adelaide

10

50%

Results

6. Assessment Outcomes

Rqt.6.1 Assessment outcomes should align with educational values

Rqt.6.2 Assessment outcomes should align with the students' goals

Rqt.6.3 Assessment outcomes should align with community goals

Rqt.6.4 Assessment outcomes should be justifiable (i.e. valid, consistent, reliable)

Rqt.6.5 Assessment outcomes should align with government goals

ATN Assessment 2008 Adelaide

11

50%Results

7. Feedback

Rqt.7.1Students should receive specific, constructive and developmental feedback on their learning and performance

Rqt.7.2 Feedback should be timely

Rqt.7.3 Feedback should be supportive

Rqt.7.4Assessment instruments and processes should continuously be evaluated and adjusted

ATN Assessment 2008 Adelaide

12

50%Results

8. Workloads

Rqt.8.1Assessment should be sufficient to judge a student's learning, but not excessive

Rqt.8.2 Assessment should avoid excessive workload on lecturers

Rqt.8.3 The effort involved in an assessment task should match the value awarded

9. Standards

Rqt.9.1 Assessment should uphold academic standards (accredited)

Rqt.9.2 Assessment should uphold ethical standards

Rqt.9.3 Assessment outcomes should be confidential

ATN Assessment 2008 Adelaide

13

50%

Results

Discussion

Many of the principles of good assessment discussed in the literature (summarised by Suskie, 2006) are included here

It provides a framework for universities to question their own principles of assessment

It provides a framework for future refinement

ATN Assessment 2008 Adelaide

14

The top ten principles are focussed on enhancing student engagement –

e.g. Assessment should… return ‘specific and constructive’ feedback be fair and equitable judge learning outcomes

The next 2 principles describe most important constraints – continuous evaluation and revision of assessment uphold academic standards

ATN Assessment 2008 Adelaide

15

Discussion

The next 2 (13th and 14th) address student engagement, but may be thought implicit: assessment should be progressive assessment tasks should be clear

Only at 15th do we have – assessment should address Graduate Qualities (43% support)

The next 5 address student engagement – deep learning, marks aligned with effort, valid criteria, participatory assessment, and learning outcome variety

ATN Assessment 2008 Adelaide

16

Discussion

Lowest support (11% to 29%) addressed: technical stuff – values, standards, goals workloads clarity & meaningfullness of tasks relation between criteria and learning outcomes supportive feedback design to minimise plagiarism

ATN Assessment 2008 Adelaide

17

Discussion

Conclusions & Open Questions

We present a framework of university supported principles for good assessment

Australian universities do tend to promote principles of student engagement through assessment.

But what do the differences in support mean? Inertia? Differences in expressing similar concepts? Assumptions on implicit or explicit principles? Branding?

ATN Assessment 2008 Adelaide

18

What direction are Australian universities taking in development of assessment guidelines towards better student engagement?

How does practice follow guidelines? Are assessment practices in Australian

universities ‘broken’?

ATN Assessment 2008 Adelaide

19

Conclusions & Open Questions