Scoring Rubrics.ppt

Scoring of Assessment

Rubrics
= scoring guidelines= grading criteriaA list of things youre looking for when youre grading tests, papers, or projectsOften with guidelines or standards for assessing them
*

*

What is a rubric?
A rubric is a set of scoring guidelines for evaluating student work.Rubrics answer the questions: By what criteria should performance be judged? Where should we look and what should we look for to judge performance success? What does the range in the quality of performance look like?How do we determine validly, reliably, and fairly what score should be given and what that score means? How should the different levels of quality be described and distinguished from one another?

What is a Rubric?
Guidelines by which a product is judgedExplain the standards for acceptable performance or workPoints out what is significant or important in learningCriteria for consistent evaluationGuide students towards improved performanceProvide specific feedback to students on how to improve performanceWithout a standard of evaluation, assessments are no more than instructional tasks

Why Use Rubrics?
Help students understand your expectations.Inspire better student performance.Make scoring easier & faster.Make scoring more accurate, unbiased, and consistent.Improve communication with students.Stop arguments with students.
*

*

When are rubrics used? With anything except a multiple choice test.

Anything for which you might give partial credit

What is a grading rubric?

A scoring guideline for evaluating student assignments such as essays, papers or presentations.

It describes the levels of achievement for a specific assignment.

*

Advantages:

March 28, 2008

What is a scoring rubric?
Scoring guide or scoring matrixContains well-defined and systematically applied criteriaEspecially useful for assessing behavior or a product that has a strong subjective component (that is, cannot be scored objectively)Examples: homework problems, oral presentations, essays, senior design projects, group skills, leadership.

What does a scoring rubric look like?
Rows*What is to be assessedOutcomesColumns*Levels of performance for that outcomeFrom acceptable to unacceptableScores assigned to each levelCellsDescription (text) of what constitutes each level of performance for that outcome
* Rows and columns may be switched

Types of scoring rubrics
Holistic (one dimension)To rate the overall process or product as a wholeUses a single scaleGenerates a single scoreDetailed or analytical (more than one dimension)To rate parts of the process or productSeparate rating scales for each partGenerates several scoresScores frequently summed

Types of rubrics
Analytic rubricProvides separate scores for specific components to be measured.Holistic rubricProvides a single overall score for the assignment.
March 28, 2008

*

Analytic Rubric
Describes levels of performance for each criterion to assess student performance on each of them.

Analytical Rubric - Research
Levels of PerformanceCriteriaWeight123Number of Sources x11-4 5-910-12Historical Accuracy x3Lots of historical inaccuracies Few inaccuracies No apparent inaccuracies Organizationx1Can not tell from which source information came Can tell with difficulty where information came from Can easily tell which sources info was drawn from Bibliography x1Bibliography contains very little information Bibliography contains most relevant information All relevant information is included

Holistic Rubric
Assigns a level of performance by assessing performance across multiple criteria as a whole. Does not list separate levels of performance for each criterion.

Holistic Rubric - Research

Rubric

3 - Excellent Researcher
included 10-12 sources no apparent historical inaccuracies can easily tell which sources information was drawn from all relevant information is included
2 - Good Researcher
included 5-9 sources few historical inaccuracies can tell with difficulty where information came from bibliography contains most relevant information
1 - Poor Researcher
included 1-4 sources lots of historical inaccuracies cannot tell from which source information came bibliography contains very little information

Advantages
Can be used to grade complex products or behaviors efficiently. A single grader can score assignments by different students more consistently.Multiple graders can score consistently.Communicates expectations so students turn in a better product.Make rationale for score transparent (reduces arguments over grades)
March 28, 2008

*

Disadvantages
May not be able to address everything we want students to know.May limit student response (to only what is defined by the rubric)May limit creativity.Time-consuming to create.
March 28, 2008

*

Advantages of using rubrics

For the Professor
Allow evaluation and assessment to be more objective and consistent Help focus to clarify his/her criteria in specific terms Provide useful feedback regarding the effectiveness of the instruction Provide benchmarks against which to measure and document progress
For Students
Help them define "qualityPromote student awareness of about the criteria to use in assessing peer performanceHelp students judge and revise their own work before handing in their assignments. Clearly show the student how their work will be evaluated and what is expected

What does a rubric need to contain?
1. Contains a scale of possible points to be assigned in scoring work, on a continuum of quality. High numbers usually are assigned to the best performances: scales typically use 4, 5 or 6 as the top score, down to 1 or 0 for the lowest scores in performance assessment.2. Provides descriptors for each level of performance to enable more reliable and unbiased scoring.

Cont.
3. Is either holistic or analytic. If holistic, a rubric has only one general descriptor for performance as a whole. If analytic, there are multiple rubrics corresponding to each independent dimension of performance being scored.

Criteria for Rubrics
Descriptors, traits, characteristics are terms used to indicate the elements contained in rubrics.Reflect the elements of good performance and become the source for the criteria used to judge the performance.Three types of criteria to determine grades:Product, Process, Progress

Product Criteria
What students produce (pamphlet, brochure, play a game, create a game, perform a routine).Performance and projects are examples of products.Rubric contains critical elements necessary for required content.Stating performance outcomes in terms of products says nothing about application or correct form.These are process criteria.

Process Criteria
Critical elements necessary for correct performance.Outlines the processes that students use to learn.In Physical Education process criteria refers to the quality of physical performance.Self-check, peer evaluation, teacher evaluation.Evaluating process says nothing about how much a student has improved.These are progress criteria.

Progress Criteria
Determine how much a student has improved or progressed.Determines the mastery of the critical elements of performance.In order to measure progress teachers must administer assessments more than once.

Types of Rubrics
The type of rubric chosen for assessment depends on the task being evaluated and the needs of the assessor.Analytic and holistic rubrics are very complex and require time to generate.Checklists are the easiest to create.Point System rubrics are easy to convert to grades.

Steps to designing a rubric

Using your learning objectives as a guide, determine the components you want to assess.

Choose the type of rubric you want to create.

Clearly identify and describe levels of performance.

Develop a scoring scale.

Test the rubric.

Revise as needed.

*

March 28, 2008

Example: Communications objectives

Students will be able to:
Demonstrate their knowledge of different criticisms of the journalism fieldDevelop a literature review that is consistent with research protocolCollect and analyze data pertaining to a research topicPresent findings from a research studyDiscuss the implications of the findings of a research study
*

March 28, 2008

Communications Assignment Rubric

*

March 28, 2008

Grammar and Quality of Writing

Development of Research Question

Explanation of Data Collection Method

Presentation of Findings

Discussion of the Implications of the Findings


*

March 28, 2008


Proper use of spelling, punctuation and grammar.

Typed using appropriate format.

Writing is clear and well organized

Frequent error in grammar, spelling and punctuation.

Formatting was followed but paper-contained typos.

Writing is unclear.


Research question is well developed. Question allowed researcher to gather detailed information on topic

Research question poorly developed. Did not help researcher in gather detailed information on topic


Provided a very descriptive and detailed account of how data was collected.

Explanation of how data was collected was insufficient.


Presented findings in a sophisticated, clear and concise manner.

Presentation of findings was not well developed and not clearly presented.


Discussion of implications of the research was very well developed and through. Recommendations for future research provided.

Implications of findings not discussed.


*

March 28, 2008





Occasional difficulties with the proper use of spelling, punctuation and grammar.

Appropriate format used.

Writing is generally clear and organized



Writing is unclear.



Research question allowed researcher to gather information on topic but could have been more concise




Provided some detail about how data was collected but lacked depth in the description of the process used to collect data.




Presented findings in a clear manner but lacked depth.




Discussion of implications of the research was presented but not well developed.


Communications Assignment Rubric (analytic)

*

March 28, 2008
Excellent ( 3)
Good (2)

Poor (1)










Writing is unclear.

















Communications Assignment Rubric (weighted analytic)

*

March 28, 2008
Excellent ( 3)
Good (2)

Poor (1)


____ x 1 = ______









Writing is unclear.


____ x 1 = ______





____ x 1.5 = _____





____ x 2 = ______





____ x 3 = ______




Example: Checklist Rubric

YesNo

________Plays within the rules of the game.

________Does not argue with others.

________Respects the other teams efforts.

________Accepts the calls of officials.
8.bin

Example: Self-Assessment

____I play within the rules of the game.

____I dont argue with others.

____I offer encouragement and support.

____I accept the outcome of the game.

Point System Rubrics
Similar to checklists they award point for the various criteria on the list.Judgment of quality is not required.Provides students with feedback.Points can be added up and converted to a grade.Certain traits or characteristics can be weighted giving them greater emphasis.9.bin

Example: Scoring Rubric

Fitness Portfolio

___ Fitness Evaluation (8 points)

___ Cardiovascular Assessment (4 points)

___ Abdominal Endurance (2 points)

___ Flexibility Assessment (2 points)

___ Fitness Plan (18 points)

___ Calculate target heart rate (2 points)

___ Needs analysis (4 points)

___ Workout Plan (12 points)

___ Warm-up and recording chart (3 pts + 1 point for chart)

___ Aerobic plan and chart (3 pts + 1 point for chart)

___ Strength plan and chart (3 pts + 1 point for chart)

Analytic Rubrics
Requires the scorers judgment to determine the degree of quality.Evaluate the strength or weakness of a trait or element.Students can easily see areas on which they must improve.These rubrics take more time to score.Words give a verbal indication of the degree of quality.Always Frequently Sometimes Never 4 3 2 1

1234

1234

1234

1234

1234

1234

Quantitative Analytic Rubric

Game Play Assessment for Tennis

The Student:
Uses correct form on the forehand.Places shots.Moves into position quickly.Returns to base position.Calls shots correctly.Knowledge of the rules is evident.
Never Sometimes Frequently Always

Useful for providing feedback to students and assessing their ability to utilize skills in an applied setting.

Qualitative Analytic Rubric

Rules for Tennis
Is unfamiliar with the rules. Depends on opponents or partner for instruction. Struggles with most questions. Is unsure of serving order and rotation. Shows some evidence of knowing rules. Struggles with some questions. Serving order and rotation are correct. May have a few errors.Shows evidence of usually knowing and applying rules. Can answer most questions when asked. Serving order and rotation are correct.Shows evidence of thoroughly knowing and applying rules. Can answer any question when asked. Serving order and rotation are correct.
Provide verbal descriptions of teacher expectations and are useful for providing formative feedback about several elements important to playing a game or performing well.

Holistic Rubrics Useful for Summative Evaluations

Proficient Level

Consistently Demonstrates the ability to select and successfully use the appropriate skills. Demonstrates good skill form. Moves to cover the appropriate space. Anticipates the offensive and defensive play and selects the appropriate response. Applies and follows the appropriate rules and scoring. Follows the rules of etiquette and fair play.

Competent Level

Demonstrates the ability to select and use the appropriate skills the majority of the time. Demonstrates good skill form a majority of the time. Attempts to move to cover the appropriate space a majority of the time. Selects the appropriate offensive and defensive strategy and the appropriate response most of the time. Consistently applies and follows the appropriate rules and scoring. Demonstrates good sporting behaviour consistently and follows game etiquette most of the time.

Novice Level

Inconsistently selects and uses appropriate skills. Frequently demonstrates incorrect skill form. Does not move to cover the appropriate space on the court. Little evidence of applying the appropriate strategy and response to a situation. Applies and follows appropriate rules and scoring. Inconsistent demonstration of rules of etiquette or good sporting behaviour.

Considerations to Address when Creating Rubrics
Validity Does the rubric measure what it claims to measure.Reliability Does the assessment consistently produce the same results.Transparency Criteria are clear enough to students so that they can assess themselves and others with roughly the same reliability.Subjectivity The amount of judgment used to assign a score to a students performance.

Rubric Hints and Guidelines
Use samples of student work - samples show what is possible but should not limit the potential for excellence.Share rubrics with students - criteria helps students understand teacher expectations.Let students create some rubrics - students are more likely to understand important criteria.Allow for multiple correct answers - sometimes the correct response is it depends.Frequency is not the sole indicator of quality too often when frequency is included in a rubric, the quality of the performance is not addressed.

Rubric Hints and Guidelines
Limit the scope of the assessment complex rubrics are cumbersome and often capture non-critical elements.Consider the levels of difficulty give credit for attempting more difficult tasks.Determine the number of levels to develop the more levels, the smaller the difference between levels, and reliability decreases.Adjust the rubric after, not during the assessment changing the rules while the game is being played is not fair practice.
G

r

a

mm

a

r

an

d

Qua

l

ity

o

f

W

r

i

ting

Deve

l

op

m

ent

of

Resea

r

ch

Question

E

x

p

l

anation

o

f

Data Co

ll

ectio

n

M

ethod

Pr

esentation

o

f

F

indings

Discussion o

f

the

I

m

p

l

ications

of

th

e

F

indings

G

r

a

mm

a

r

an

d

Qua

l

ity

o

f

W

r

i

ting

Pr

o

per

use

o

f

spe

lli

n

g

,

pu

n

c

t

ua

t

i

o

n

a

n

d

gra

mm

ar.

T

y

ped

us

i

n

g

appr

o

p

r

i

a

t

e

f

o

r

m

a

t

.

W

r

i

t

i

n

g

i

s

c

l

ea

r

a

n

d

we

ll

o

rga

n

i

zed

F

reque

n

t

err

o

r

i

n

gra

mm

a

r

,

spe

lli

n

g

a

n

d

pu

n

c

t

ua

t

i

o

n

.

F

o

r

m

a

tt

i

n

g

was

f

o

ll

o

wed

b

u

t

paper

-

c

o

n

t

a

i

n

ed

t

y

p

o

s.

W

r

i

t

i

n

g

i

s

u

n

c

l

ea

r

.

Deve

l

op

m

ent

of

Resea

r

ch

Question

Researc

h

ques

t

i

o

n

i

s

we

ll

de

v

e

l

o

ped.

Ques

t

i

o

n

a

ll

o

wed

researc

h

er

to

ga

t

h

e

r

de

t

a

il

ed

i

n

f

o

r

m

a

t

i

o

n

o

n

to

p

i

c

Researc

h

ques

t

i

o

n

p

oo

r

ly

de

v

e

l

o

ped.

D

i

d

n

o

t

h

e

l

p

researc

h

er

i

n

ga

t

h

er

de

t

a

il

e

d

i

n

f

o

r

m

a

t

i

o

n

o

n

to

p

i

c

E

x

p

l

anation

o

f

Data Co

ll

ectio

n

M

ethod

Pr

o

v

i

ded

a

v

er

y

descr

i

p

t

i

v

e

a

n

d

de

t

a

il

e

d

acc

o

u

n

t o

f

h

o

w

da

ta

was

c

o

ll

ec

t

ed.

E

x

p

l

a

n

a

t

i

o

n

o

f h

o

w

da

t

a

was

c

o

ll

ec

t

ed

was

i

n

su

ffi

c

i

e

n

t

.

Pr

esentation

o

f

F

indings

Prese

n

t

ed

fi

n

d

i

n

gs

i

n

a

s

o

p

h

i

s

t

i

ca

t

ed,

c

l

ear

a

n

d

c

o

n

c

i

se

m

a

nn

er.

Prese

n

t

a

t

i

o

n

o

f

fi

n

d

i

n

gs

was

n

ot

we

ll

de

v

e

l

o

ped

a

n

d

n

o

t

c

l

ear

ly

prese

n

t

ed.

Discussion o

f

the

I

m

p

l

ications

of

th

e

F

indings

D

i

scu

s

s

i

o

n

o

f

im

p

li

ca

t

i

o

n

s

o

f

t

h

e

researc

h

was

v

er

y

we

ll

de

v

e

l

o

ped

a

n

d

t

hr

o

ug

h

.

Rec

o

mm

e

n

da

t

i

o

n

s

f

o

r

f

u

t

ur

e

researc

h

pr

o

v

i

ded.

I

m

p

li

ca

t

i

o

n

s

o

f

fi

n

d

i

n

gs

n

o

t

d

i

scus

s

ed.

G

r

a

mm

a

r

an

d

Qua

l

ity

o

f

W

r

i

ting

Pr

o

per

use

o

f

spe

lli

n

g

,

pu

n

c

t

ua

t

i

o

n

a

n

d

gra

mm

ar.

T

y

ped

us

i

n

g

appr

o

p

r

i

a

t

e

f

o

r

m

a

t

.

W

r

i

t

i

n

g

i

s

c

l

ea

r

a

n

d

we

ll

o

rga

n

i

zed

Occas

i

o

n

a

l

d

iffi

cu

l

t

i

es

w

i

t

h

t

h

e

pr

o

per

use

o

f

spe

lli

n

g,

pu

n

c

t

ua

t

i

o

n

a

n

d

gra

mm

a

r

.

A

ppr

o

pr

i

a

t

e

f

o

r

m

a

t

u

sed.

W

r

i

t

i

n

g

i

s

ge

n

era

ll

y

c

l

ear

a

n

d

o

rga

n

i

zed

F

reque

n

t

err

o

r

i

n

gra

mm

a

r

,

spe

lli

n

g

a

n

d

pu

n

c

t

ua

t

i

o

n

.

F

o

r

m

a

tt

i

n

g

was

f

o

ll

o

wed

b

u

t

paper

-

c

o

n

t

a

i

n

ed

t

y

p

o

s.

W

r

i

t

i

n

g

i

s

u

n

c

l

ea

r

.

Deve

l

op

m

ent

of

Resea

r

ch

Question

Researc

h

ques

t

i

o

n

i

s

we

ll

de

v

e

l

o

ped.

Ques

t

i

o

n

a

l

l

o

wed

researc

h

er

to

ga

t

h

e

r

de

t

a

il

ed

i

n

f

o

r

m

a

t

i

o

n

o

n

to

p

i

c

Researc

h

ques

t

i

o

n

a

ll

o

wed

researc

h

er

to

ga

t

h

er

i

n

f

o

r

m

a

t

i

o

n

o

n

to

p

i

c

b

u

t

c

o

u

l

d

h

a

v

e

b

ee

n

m

o

re

c

o

n

c

i

se

Researc

h

ques

t

i

o

n

p

oo

r

ly

de

v

e

l

o

ped.

D

i

d

n

o

t

h

e

l

p

researc

h

er

i

n

ga

t

h

er

de

t

a

il

e

d

i

n

f

o

r

m

a

t

i

o

n

o

n

to

p

i

c

E

x

p

l

anation

o

f

Data Co

ll

ectio

n

M

ethod

Pr

o

v

i

ded

a

v

er

y

descr

i

p

t

i

v

e

a

n

d

de

t

a

il

e

d

acc

o

u

n

t o

f

h

o

w

da

ta

was

c

o

ll

ec

t

ed.

Pr

o

v

i

ded

s

o

m

e

de

t

a

il

a

b

o

u

t

h

o

w

da

t

a

was

c

o

ll

ec

t

ed

b

u

t

l

acked

dep

t

h

i

n

t

h

e

descr

i

p

t

i

o

n

o

f

t

h

e

p

r

o

cess

used

to

c

o

ll

ec

t

da

t

a.

E

x

p

l

a

n

a

t

i

o

n

o

f h

o

w

da

t

a

was

c

o

ll

ec

t

ed

was

i

n

su

ffi

c

i

e

n

t

.

Pr

esentation

o

f

F

indings

Prese

n

t

ed

fi

n

d

i

n

gs

i

n

a

s

o

p

h

i

s

t

i

ca

t

ed,

c

l

ear

a

n

d

c

o

n

c

i

se

m

a

nn

er.

Prese

n

t

ed

fi

n

d

i

n

gs

i

n

a

c

l

ear

m

a

nn

er

b

u

t

l

acked

dep

t

h.

Prese

n

t

a

t

i

o

n

o

f

fi

n

d

i

n

gs

was

n

ot

we

ll

de

v

e

l

o

ped

a

n

d

n

o

t

c

l

ear

ly

prese

n

t

ed.

Discussion o

f

the

I

m

p

l

ications

of

th

e

F

indings

D

i

scu

s

s

i

o

n

o

f

im

p

li

ca

t

i

o

n

s

o

f

t

h

e

researc

h

was

v

er

y

we

ll

de

v

e

l

o

ped

a

n

d

t

hr

o

ug

h

.

Rec

o

mm

e

n

da

t

i

o

n

s

f

o

r

f

u

t

ur

e

researc

h

pr

o

v

i

ded.

D

i

scu

s

s

i

o

n

o

f

im

p

li

ca

t

i

o

n

s

o

f

t

h

e

researc

h

was

prese

n

t

ed

b

u

t

n

ot

we

ll

de

v

e

l

o

p

ed.

I

m

p

li

ca

t

i

o

n

s

o

f

fi

n

d

i

n

gs

n

o

t

d

i

scus

s

ed.

Excellent ( 3) Good (2) Poor (1)

Grammar and

Quality of Writing

Proper use of spelling, punctuation and

grammar.



Occasional difficulties with the

proper use of spelling,

punctuation and grammar.


Writing is generally clear and

organized

Frequent error in grammar,

spelling and punctuation.

Formatting was followed but

paper-contained typos.

Writing is unclear.

Development of

Research Question

Research question is well developed.

Question allowed researcher to gather

detailed information on topic

Research question allowed

researcher to gather

information on topic but could

have been more concise

Research question poorly

developed. Did not help

researcher in gather detailed

information on topic

Explanation of

Data Collection

Method

Provided a very descriptive and detailed

account of how data was collected.

Provided some detail about

how data was collected but

lacked depth in the description

of the process used to collect

data.

Explanation of how data was

collected was insufficient.

Presentation of

Findings

Presented findings in a sophisticated, clear

and concise manner.

Presented findings in a clear

manner but lacked depth.

Presentation of findings was

not well developed and not

clearly presented.

Discussion of the

Implications of the

Findings

Discussion of implications of the research

was very well developed and through.

Recommendations for future research

provided.

Discussion of implications of

the research was presented but

not well developed.

Implications of findings not

discussed.

Excellent ( 3) Good (2) Poor (1)

Grammar and

Quality of Writing

____ x 1 = ______

Proper use of spelling, punctuation and

grammar.



Occasional difficulties with the

proper use of spelling,

punctuation and grammar.


Writing is generally clear and

organized

Frequent error in grammar,

spelling and punctuation.

Formatting was followed but

paper-contained typos.

Writing is unclear.

Development of

Research Question

____ x 1 = ______

Research question is well developed.

Question allowed researcher to gather

detailed information on topic

Research question allowed

researcher to gather

information on topic but could

have been more concise

Research question poorly

developed. Did not help

researcher in gather detailed

information on topic

Explanation of

Data Collection

Method

____ x 1.5 = _____

Provided a very descriptive and detailed

account of how data was collected.

Provided some detail about

how data was collected but

lacked depth in the description

of the process used to collect

data.

Explanation of how data was

collected was insufficient.

Presentation of

Findings

____ x 2 = ______

Presented findings in a sophisticated, clear

and concise manner.

Presented findings in a clear

manner but lacked depth.

Presentation of findings was

not well developed and not

clearly presented.

Discussion of the

Implications of the

Findings

____ x 3 = ______

Discussion of implications of the research

was very well developed and through.

Recommendations for future research

provided.

Discussion of implications of

the research was presented but

not well developed.

Implications of findings not

discussed.

Scoring Rubrics.ppt

Documents

acceptable performance

quality of performance

set of scoring guidelines

grading rubric

level of performance

performance success

better student performance

help students